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SECTION  1 
INTRODUCTION 

Recent  tragic  events  in  the  Persian  Gulf  drew  shaip  attention  to  human  biases  and 
cognitive  limitations  and  underscored  the  need  to  better  understand  the  forces  that  impact  deci¬ 
sionmaking  and  cognitive  processes.  These  events  also  brought  to  the  forefront  the  need  to 
examine  another  variable  that  can  have  profound  effects  on  human  judgment  —  stress.  The 
Persian  Gulf  incident  profoundly  demonstrated  the  potential  for  serious  errors  of  judgment 
among  military  personnel  operating  under  conditions  of  high  situational  and  cognitive  stress. 

In  the  case  of  the  Vincennes,  the  crew  appeared  to  exhibit  a  classic  case  of  confirmatory  bias, 
where  all  gathered  information,  regardless  of  its  true  disposition,  was  construed  as  supporting 
a  predisposed  hypothesis  of  attack.  The  crew  was  so  sure  the  enemy  would  attack  that  they 
intexpreted  all  incoming  information  in  tight  of  that  bias.  Thus,  all  information  appeared  to 
confirm  an  attack.  This  "processed"  interpretation  of  the  data  was  the  only  interpretation 
transmitted  to  the  captain.  The  captain  sought  and  considered  only  the  interpreted  data  of  his 
crew  and  did  not  seek  any  raw  measurements  of  his  own,  even  though  this  was  possible. 
Based  on  the  available  information,  the  captain  acted  as  only  he  could,  and  took  hostile  actions. 

Under  more  normal  conditions  such  actions  would  be  extremely  unlikely.  More  data 
correlation  would  occur,  some  crew  members  would  challenge  the  attack  bias,  and  other  pos¬ 
sible  hypotheses  and  interpretations  on  the  data  would  be  entertained.  More  than  likely,  the 
captain  would  call  for  some  corroborating  independent  raw  measurements.  But,  everything 
was  not  normal.  The  attacks  earlier  that  day  rested  heavily  on  everyone's  mind  as  did  the  inci¬ 
dent  involving  the  Stark  where  inaction  led  to  tragedy  and  disgrace.  These  thoughts  worried 
and  distracted  the  crew;  in  addition,  many  of  the  crew  feared  for  their  safety.  There  was  also  a 
strong  impression  that  decision  time  was  short  and  growing  ever  shorter  and  that  a  decision 
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must  be  made.  These  are  the  classic  ingredients  of  stress.  We  suspect  that  stress  was  one  of 
the  major  contributors  to  the  crew's  errors  in  judgment.  Stress  can  impair  a  decisionmaker's 
normal  patterns  of  rational,  coherent,  and  flexible  thinking.  It  also  appears  that  under  stress, 
an  individual's  immediate  memory  span  is  reduced,  thinking  becomes  more  simplistic,  there  is 
difficulty  dealing  conceptually  with  as  many  categories  as  when  unstressed,  and  there  is  a  fail¬ 
ure  to  recognize  all  the  options  available  and  to  adequately  evaluate  these  alternatives  (Janis  and. 
Mann,  1977). 

1.1  RESEARCH  GOAL 

The  ultimate  objectives  of  this  research  effort  are  to  improve  the  understanding  of 
information  processing  and  decisionmaking  that  arise  in  the  context  of  command  and  control 
of  naval  battle  forces  and  to  develop  a  quantitative  methodology  for  designing  superior  organ¬ 
izational  structures  that  support  these  decision  processes.  The  specific  goal  of  the  research  is 
to  investigate  decisionmaking  and  information  seeking  processes  in  a  difficult  and  stressful 
environment.  The  research  makes  the  following  contributions: 

•  Provides  quantitative  results  on  how  the  processes  of  decisionmaking  and 
information  seeking  are  altered  by  increases  in  cognitive  stress. 

•  Provides  corroborating  findings  for  an  earlier  study  (Kastner  et  al.,  1989) 
examining  decisionmaking  and  information  seeking. 

•  Identifies  information-seeking  strategies  and  analyzes  the  errors  associated 
with  each. 

•  Provides  a  comparison  between  civilian  and  military  decisionmakers. 

1.2  PAST  EXPERIMENTAL  EFFORT 

Kastner  et  al.  (1989)  examined  the  effects  of  task  difficulty,  information  cost,  consul¬ 
tant  expertise,  and  consultant  membership  (i.e.,  team  member  or  independent)  on  the  processes 
of  information  gathering  and  decisionmaking.  Each  of  eight  subjects  assumed  the  role  of  an 
antisubmarine  warfare  commander  and  decided  if  a  detected  submarine  was  hostile  or  friendly. 
The  decision  was  based  on  the  pump  noise  signature  of  the  detected  submarine.  Commanders 
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were  aware  that  the  friendly  submarine's  pump  noise  signature  was  370  Hz,  the  hostile  sub¬ 
marine’s  pump  noise  signature  was  either  330  Hz  or  360  Hz,  and  the  detection  sensors  had  an 
inherent  error  distribution  with  a  standard  deviation  of  20  Hz.  The  graphical  display  used  by 
the  subjects  is  depicted  in  Fig.  1-1.  To  aid  in  the  discrimination,  decisionmakers  could  probe 
(task  their  sensors  for  another  measurement)  or  ask  the  opinion  of  a  knowledgeable  consultant. 
Information  requests  always  incurred  a  cost.  According  to  the  experimental  condition,  the 
consultant  could  be  a  superior  expert  to  the  decisionmaker  (i.e.,  receive  better  information  than 
the  decisionmaker)  or  equal  to  the  decisionmaker  in  expertise.  Also  by  condition,  the  consul¬ 
tant  was  presented  as  a  team  member,  where  he  was  aware  the  decisionmaker  had  asked  for 
help  and  had  some  knowledge  of  the  commander's  decisionmaking  process,  or  as  an  indepen¬ 
dent,  where  he  had  no  knowledge  of  the  decisionmaker  or  his  request  for  help. 

T  0  S  C  fl 

Condition  Code:  2221  Trial  Number:  1  Name:  TRAINING2 

Pump  Noise 
Enemy  Sub  330  Hz 
Friendly  Sub  370  Hz 

Sensor  Standard  Deviation 
Yours  20  Hz 

Consultant's  20  Hz 

Command  Area 

initial  Measurement^  382  Hz 


R-7135 

Figure  1-1.  Graphical  Display  Used  by  Subjects  in  Kastner  et  al.. 
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To  help  guide  the  experimental  design  and  hypothesis  generation,  as  well  as  model  the 
decisionmaking  task  confronting  the  decisionmakers,  Kastner  et  al.  developed  the  team  optimal 
Stopping  with  communication  alternatives  (TOSCA)  normative  model.  The  TOSCA  model  is 
based  on  an  earlier  model  by  Papastavrou  and  Athans  (1986)  that  is  extended  to  add  the  probe 
option.  Specifically,  TOSCA  addresses  under  what  circumstances  decisionmakers  should  stop 
gathering  information  and  start  making  decisions,  and  under  what  conditions  decisionmakers 
should  seek  processed  information  (consultant  opinions)  rather  than  raw  information  (probe 
measurements).  The  problem  that  is  solved  by  the  decisionmakers  and  the  TOSCA  model  is 
graphically  depicted  in  Fig.  1-2. 


Figure  1-2.  The  Decisionmaking  Problem  Addressed  by  the  TOSCA  Model. 

Figure  1-2  shows  that  given  prior  information  and  a  measurement,  the  decisionmaker 
must  choose  whether  to  stop  gathering  information  and  select  Ho  or  Hi  as  correct,  or  seek 
more  information.  If  the  decision  is  to  collect  more  information,  then  the  decisionmaker  must 
choose  whether  to  probe  for  a  second  measurement  or  ask  for  the  consultant's  opinion.  This 
new  information  is  then  folded  into  the  decisionmaker’s  current  estimate  to  allow  selection  of 
Ho  or  Hi.  The  optimal  decision  rules  employed  by  the  TOSCA  model  to  govern  these  deci¬ 
sions  are  characterized  as  threshold  strategies.  Figure  1-3  shows  how  the  thresholds  are 
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derived  from  a  set  of  intersecting  expected  cost  curves,  which  are  derived  from  detection  theory 
(Van  Trees,  1968)  and  team  theory  (Marschak  and  Radner,  1972).  One  curve  determines  the 
cost  of  stopping  and  deciding  Hi,  while  another  curve  computes  the  cost  of  stopping  and 
selecting  Ho-  Expected  cost  curves  and  the  resulting  thresholds  are  also  computed  for  probing 
and  for  asking  the  consultant.  Each  threshold  delineates  a  decision  region.  For  example,  if 
the  initial  measurement  falls  left  of  the  left-most  threshold  (see  Fig.  1-3),  the  model  specifies 
that  the  best  strategy  is  to  stop  gathering  information  and  decide  Hi.  Similarly,  if  the  initial 
measurement  is  to  the  right  of  the  right-most  threshold,  the  optimal  decision  is  to  stop  and 
decide  Ho.  If  the  initial  measurement  lies  in  the  region  between  the  two  left  thresholds  or  in 
the  region  between  the  two  right  thresholds,  then  the  optimal  strategy  is  to  probe  for  another 
measurement.  In  other  words,  the  cost  of  probing  is  worth  the  expected  information  gain. 
Finally,  if  the  initial  measurement  falls  in  the  region  between  the  two  innermost  thresholds, 
then  the  best  strategy  is  to  ask  the  consultant's  opinion.  That  is,  the  expected  information  gain 
is  worth  the  cost  of  asking  the  consultant. 


Figure  1-3.  Normative  Model:  Illustration  of  Threshold  Strategies. 
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Drawing  on  the  TOSCA  model  and  cognitive  psychology,  the  researchers  proposed  and 
tested  a  number  of  hypotheses.  Some  of  the  confirmed  hypotheses  are  that  decisionmakers 
would:  choose  between  probing  and  consulting  using  information  cost-benefit  rules;  ask  the 
consultant  more  often  when  depicted  as  a  team  member;  exhibit  threshold  type  choice  strategies 
(similar  to  that  of  the  normative  model);  and  all  things  being  equal,  favor  probing  to  consulting, 
because  a  probe  measurement  provides  more  bits  of  information.  The  main  hypothesis  that 
was  not  confirmed  predicted  decisionmakers  would  not  seek  additional  information  as  often  in 
difficult  discrimination  conditions.  Just  the  opposite  was  observed,  more  requests  for  infor¬ 
mation  were  made  in  the  difficult  condition. 

In  terms  of  performance,  subjects  did  very  well;  in  fact,  they  approached  the  values 
obtained  by  the  normative  model.  The  subjects  achieved  this  performance,  however,  using 
strategies  different  from  those  indicated  by  the  TOSCA  model. 

Discussing  the  experimental  results,  Kastner  et  al.  noted  that  decisionmakers  were 
inclined  to  seek  more  information  than  predicted  by  the  TOSCA  model,  and  this  was  partic¬ 
ularly  true  for  difficult  discrimination  conditions.  When  information  was  sought,  decision¬ 
makers  favored  the  processed  judgments  of  the  consultant  over  the  raw  measurements  provided 
by  a  probe.  Decisionmakers,  moreover,  sought  information  even  when  the  benefit  of  that 
information  was  n<2l  worth  the  cost  of  obtaining  it. 

These  findings  were  obtained  under  relatively  low-stress  conditions.  There  was  no 
time  pressure  and  relatively  few  distractors.  Although  the  subjects  tended  to  perform  well, 
their  strategies  were  nonoptimal  and  in  some  conditions  (e.g.,  gathering  information  when  it 
was  not  worth  it)  decidedly  counterproductive.  If  these  subjects  were  thrust  into  a  stressful 
situation,  how  might  their  strategies  change?  Would  stress  promote  further  nonoptimal 
behavior  and  exacerbate  counterproductive  strategies?  How  would  performance  be  affected? 
Will  stress  foster  more  false  alarms?  These  are  some  of  the  questions  the  current  experiment 
was  designed  to  address,  building  upon  the  TOSCA  experimental  paradigm. 
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SECTION  2 
STRESS 

Stress  is  a  ubiquitous  part  of  everyday  life.  It  is  an  inevitable  part  of  the  challenges  that 
prompt  mastery  of  new  skills  and  behaviors.  Problems  arise  when  stress  becomes  excessive. 
The  individual  then  experiences  disrupted  emotional,  cognitive,  and  physiological  functioning. 
The  impact  of  cognitively  induced  stress  on  information  processing  and  decisionmaking  pro¬ 
cesses  is  the  primary  focus  of  this  effort,  thus,  it  behooves  us  to  define  and  operationalize  what 
we  mean  by  stress. 

2.1  THREE  APPROACHES  TO  STRESS 

Conceptually,  stress  is  a  relatively  new  issue  in  psychological  research  and  there  is 
some  disagreement  as  to  its  definition.  Cider  et  al.  (1983)  describe  three  approaches  or  models 
of  stress  that  might  be  informative  in  arriving  at  an  understanding  and  definition.  The 
response-based  model  of  stress  holds  that  stress  is  a  cluster  of  disturbing  psychological  and 
physiological  responses  to  difficult  situations.  A  second  approach,  the  stimulus-based  model, 
views  stress  in  terms  of  environmental  events  or  stimuli  that  cause  such  responses.  The  inter¬ 
actional  model  of  stress  (Cox,  1978)  approaches  stress  as  an  imbalance  between  individual 
needs  and  abilities  on  the  one  hand  and  environmental  demands  on  the  other. 

A  prime  contributor  to  the  response-based  model  is  Hans  Selye  and  his  concept  of  the 
General  Adaptation  Syndrome  or  GAS  (Selye,  1973, 1976).  The  GAS  is  a  pattern  of  physio¬ 
logical  responses  that  describes  three  stages  of  the  body's  resistance  to  prolonged  stressors. 
The  three  stages  —  alarm,  resistance,  and  exhaustion  —  describe  how  the  body  rapidly 
mobilizes  its  physiological  defenses,  tries  to  adapt  to  the  presence  of  the  stressor,  and  finally 
exhausts  its  adaptive  energy  if  the  stressors  persist  The  major  strength  of  the  response- based 
model  is  its  demonstration  that  many  different  stressors  produce  a  similar  syndrome  or 
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response  (Cider  et  al.,  1983).  Cider  et  al.  note  that  this  approach  does  not,  however,  describe 
the  characteristics  of  stressors.  To  predict  if  a  certain  event  will  produce  a  stress  syndrome,  an 
understanding  of  the  characteristics  of  stressful  stimuli  is  needed. 

The  stimulus-based  model  focuses  on  three  important  characteristics  of  stressful 
stimuli:  1)  overload,  2)  conflict,  and  3)  uncontrollability.  When  a  stimulus  becomes  so 
intense  that  an  individual  can  no  longer  adapt  to  it,  overload  occurs.  An  important  form  of 
overload  is  work  overload.  Ivancevich  and  Matteson  (1980)  discuss  two  different  kinds  of 
work  overload:  when  there  are  too  many  things  to  do  in  too  little  time  and  when  performance 
standards  are  so  high  that  work  cannot  be  satisfactorily  completed  no  matter  how  much  time  is 
available.  When  two  or  more  incompatible  response  tendencies  are  simultaneously  aroused, 
conflict  occurs.  Conflict  situations  are  ambiguous  in  that  the  stimulus  arousing  them  does  not 
imply  which  response  tendency  is  appropriate  (Cider  et  al.,  1983).  Research  conducted  with 
both  animals  and  people  show  that  conflict  is  capable  of  producing  powerful  stress  responses. 
Uncontrollable  situations  are  also  highly  stressful.  Studies  such  as  those  of  Weiss  (1972) 
show  that  the  stress  produced  by  aversive  stimuli  may  have  more  to  do  with  one's  ability  to 
control  the  timing  and  duration  of  these  stimuli  than  with  the  actual  pain  they  cause.  Moreover, 
the  stressful  impact  of  aversive  stimuli  can  be  lessened  if  people  merely  believe  they  have  con¬ 
trol,  as  demonstrated  by  the  work  of  Glass  and  Singer  (1972). 

A  major  asset  of  the  stimulus-based  model  is  in  helping  predict  the  sorts  of  situations 
individuals  will  find  stressful.  The  implication  is  that  stress  reactions  and  their  detrimental 
effects  can  be  lessened  or  eliminated  by  modifying  stimulus  characteristics  such  as  overload, 
conflict,  and  uncontrollability.  The  stimulus-based  model  is  limited  in  that  not  all  individuals 
find  the  same  stimuli  equally  stressful.  This  means  individual  differences  among  people  must 
be  taken  into  account  in  order  to  predict  whether  a  given  stressor  will  produce  a  stress 
response. 

The  interaction  model  holds  that  stress  will  occur  when  two  conditions  are  met:  the 
individual  perceives  a  threat  to  important  needs  and  motives,  and  the  individual  is  unable  to 


8 


lh  J54 


ALPHATECH,  INC. 


cope  with  the  stressor  (Cider  et  al.,  1983).  This  model  also  suggests  that  the  other  two  models 
require  bolstering  with  knowledge  about  individual  motives  and  coping  abilities.  It  is  predicted 
by  this  position  that  the  amount  of  stress  experienced  in  any  situation  will  depend  on  the 
balance  between  stressor  demands  and  coping  skills.  Where  there  is  an  imbalance  between 
the  two,  i.e.,  when  demands  exceed  coping  skills,  stress  will  be  high.  When  coping  skills 
approximate  demands,  stress  will  be  low.  For  example,  the  Mercury  astronauts  had  many 
good  reasons  to  exhibit  substantial  amounts  of  stress.  Yet,  careful  analysis  revealed  almost 
no  evidence  of  stress  reactions  during  the  flights.  A  combination  of  background,  personality, 
and  training  provided  the  astronauts  with  coping  skills  adequate  to  the  demands  of  space  flight. 
When  confronted  with  a  difficult  situation,  the  astronauts  would  pause,  assess  the  situation, 
decide  on  a  course  of  action,  and  then  follow  through  (Cox,  1978;  Wolfe,  1979). 

Accounting  for  individual  differences  in  the  experience  of  stress  is  a  primary  strength  of 
the  interaction  model.  Conversely,  the  model  says  little  about  the  nature  of  stress  reactions  or 
the  stimulus  conditions  most  likely  to  produce  such  reactions. 

Drawing  on  the  work  of  Cox  (1978),  Lazarus  (1976),  and  McGrath  (1970),  Cider  et  al. 
(1983)  offer  the  following  comprehensive  definition  of  stress,  "a  particular  pattern  of  disturb¬ 
ing  psychological  and  physiological  reactions  (response-based  model)  that  occurs  when  an 
environmental  event  (stimulus-based  model)  threatens  important  motives  and  taxes  one's 
ability  to  cope  (interaction  model)"  (p.  489). 

2.2  STRESS-INDUCED  COGNITIVE  DISRUPTIONS 

Under  stress,  particularly  high  stress,  persons  may  experience  such  an  inordinate 
amount  of  cognitive  constriction  and  perseveration  that  their  thought  processes  are  disrupted 
(Janis  and  Mann,  1977).  Normally,  during  periods  of  nonstress,  our  thinking  is  essentially 
rational,  logical,  and  flexible.  During  periods  of  stress,  thinking  is  often  dominated  by  worries 
about  the  consequences  of  our  actions  and  by  negative  self-evaluations.  One’s  normal 
patterns  of  organized,  logical,  and  coherent  thinking  are  impaired.  Individuals  in  stressful 
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environments  report  worrying  about  possible  failure  and  about  their  own  inadequacies.  This, 
in  turn,  interferes  with  thinking  about  the  tasks  they  have  to  perform  (Spielberger,  1979). 

High  levels  of  stress  disturb  individuals'  ability  to  concentrate  and  the  process  of  selective 
attention,  i.e.,  the  ability  to  concentrate  on  specific  stimuli  while  ignoring  other  irrelevant 
stimuli.  In  such  stress  situations,  persons  report  being  distracted  both  by  obsessive  thoughts 
of  failure  and  by  external  stimuli.  Obviously,  poor  concentration  impairs  an  individual's 
performance  and  decisionmaking  ability. 

Stress-induced  memory  impairment  is  most  likely  due  to  the  deterioration  of  the  ability 
to  transfer  information  from  short-term  to  long-term  memory.  The  rehearsal  process  (neces¬ 
sary  for  short-  to  long-term  transfer)  appears  to  be  disrupted  under  stress,  leading  to  frequent 
confusion  about  the  sequence  of  events  (Rimm  and  Somervill,  1977). 

Janis  and  Mann  (1977)  report  the  immediate  memory  span  of  individuals  is  reduced  and 
their  thinking  becomes  more  simplistic.  That  is,  they  cannot  deal  conceptually  with  as  many 
categories  as  when  they  are  unstressed.  People  fail  to  recognize  all  the  options  open  to  them 
and  fail  to  use  remaining  resources  to  evaluate  adequately  those  alternatives  of  which  they  are 
aware.  Under  high  stress  people  are  likely  to  search  frantically  for  a  solution,  persevere  in 
their  thinking  about  a  limited  number  of  options,  and  then  stick  tightly  to  a  hastily  contrived 
solution  that  appears  to  promise  immediate  relief. 

In  summary,  stress  is  capable  of  disrupting  a  variety  of  cognitive  functions.  This 
implies  that  decisionmaking,  which  usually  requires  a  combination  of  good  concentration, 
flexible  thinking,  intact  memory,  and  visual  imaging,  will  be  significantly  affected  by  stress. 

2.3  OPERATIONAL  DEFINITION  OF  STRESS 

Employing  the  stimulus-based  model  to  characterize  the  stress-inducing  stimuli  to  be 
manipulated,  we  focus  on  the  three  attributes  of  overload,  conflict,  and  uncontrollability.  One 
component  of  overload  that  will  be  easy  to  manipulate  in  the  experimental  paradigm  is  time 
pressure.  The  constant  worry  of  being  able  to  finish  in  the  allotted  time  is  stressful.  In  the 
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high-stress  condition,  subjects  are  informed  that  they  will  have  a  fixed  time  by  which  to  make 
their  determination  of  enemy  or  friendly.  To  keep  subjects  constantly  aware  of  the  time  limit, 
a  time-bar  is  added  to  the  display  screen.  Each  four-second  passage  of  time  is  denoted  by  a 
beep.  Subjects  who  do  not  complete  the  trial  in  the  allotted  time  receive  a  score  of  0  for  that 
trial. 

The  addition  of  a  secondary  task  to  the  experimental  paradigm  invokes  all  three  stress- 
inducing  stimuli  associated  with  the  stimulus-based  model.  The  secondary  task  cuts  deeply 
into  a  subject's  residual  resources  or  capacity  not  utilized  in  the  primary  task  (Ogden,  Levine, 
and  Eisner,  1979;  Rolfe,  1973).  Ogden  et  al.  and  Rolfe  also  note  that  a  secondary  task  can 
serve  as  a  loading  task;  thus,  decrement  variations  in  the  secondary  task  can  be  used  to  infer 
differences  in  the  primary  task  demand.  A  conflict  is  also  created  as  subjects,  at  various  times, 
have  to  decide  which  task  must  be  immediately  attended.  Subjects,  moreover,  have  little  or  no 
control  over  the  appearance  of  the  secondary  task,  thereby  creating  some  sense  of  uncontrolla¬ 
bility.  Lastly,  the  secondary  task  is  intrusive  and  distracting  to  the  performance  of  the  primary 
task. 

In  summary,  the  two  manipulation  components  of  stress  —  time  pressure  and  secondary 
task  —  are  used  to  provide  a  significant  induction  of  cognitive  stress. 
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SECTION  3 
METHOD 

3.1  OVERVIEW 

The  experimental  paradigm  is  a  modification  of  the  team-optimal  stopping  with  com¬ 
munication  alternatives  (or  TOSCA)  paradigm  employed  by  Kastner  et  al.  (1989).  The  review 
of  Kastner  et  al.  in  subsection  1.2  clearly  indicates  that  the  TOSCA  paradigm  is  reliable  and 
provided  a  viable  means  for  investigating  hypotheses  concerning  information  gathering  and 
decisionmaking  processes. 

As  an  abstraction  of  an  anti-submarine  warfare  (ASW)  detection  problem,  the  paradigm 
requires  subjects  assuming  the  role  of  an  ASW  commander  to  decide  if  a  detection  is  a  hostile 
or  friendly  submarine.  This  decision  is  based  on  the  pump  noise  signature  of  the  detected 
submarine.  Commanders  are  told  that  the  friendly  submarine's  pump  noise  signature  is  always 
370  Hz  and  the  hostile  submarine's  pump  noise  signature  is  either  330  Hz  or  360  Hz.  The 
detection  sensors  are  described  as  having  an  inherent  error  distribution  with  a  zero  mean  and 
a  standard  deviation  of  20  Hz.  To  aid  in  the  discrimination,  task  commanders  may  probe 
(i.e.,  task  their  sensors  for  another  measurement)  or  ask  the  opinion  of  a  knowledgeable 
"team-member  consultant."  Subjects  are  aware  that  either  resource  has  a  cost  associated  with 
it  and,  under  certain  conditions,  the  consultant  will  receive  better  information  than  them. 

Stress  is  induced  by  imposing  a  specific  deadline  (time  pressure)  and  requiring  the 
commanders  to  attend  to  an  intrusive  second  task.  At  the  beginning  of  every  stress  trial,  a 
vertical  time  bar  appears.  The  time  bar  starts  at  24  seconds  and  decreases  at  1 -second  intervals. 
After  the  passage  of  every  4  seconds,  a  beep  is  sounded.  At  4  seconds  into  each  stress  trial, 
the  secondary  task  window  appears.  The  secondary  task  is  presented  as  a  communication  task 
where  the  commander  is  shown  the  value  of  a  pump  noise  measurement  supposedly  taken  from 
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a  second  contact.  After  noting  its  value,  the  commander  transmits  the  detected  pump  noise 
measurement  to  another  platform.  Ten  seconds  later,  the  pump  noise  measurement  is  trans¬ 
mitted  back  for  verification.  During  a  stress  trial,  participants  must  complete  the  secondary 
task  along  with  the  primary  discrimination  task  in  the  allotted  time. 

3.2  SAMPLES 

Participants  for  the  experiment  are  drawn  from  two  distinct  populations  civilian  and 
military.  The  civilian  sample  consists  of  nine  male  and  female  volunteers  from  an  engineering 
firm  and  college  students.  The  engineering  employees  range  in  age  from  25  to  30  years,  are 
well  educated,  and  have  decisionmaking  responsibilities  within  the  company.  The  college 
students  range  in  age  from  18  to  21  years  and  are  engineering  or  science  majors.  Seven  male 
officers,  either  instructors  or  students  at  the  Naval  War  College  in  Newport,  Rhode  Island, 
comprise  the  military  sample.  All  the  military  participants  are  at  grade  0-4  or  above.  Partici¬ 
pants  work  independently  and  experience  four  to  five  hours  of  training  and  data  collection. 

3.3  VARIABLES  MANIPULATED 

Four  independent  variables  are  operationalized  and  manipulated.  Stress  varies  the 
workload  and  time  pressure  facing  the  participants.  During  low  stress,  subjects  have  no  time 
deadline  and  no  other  task  but  the  primary  discrimination  task  to  complete.  Under  conditions 
of  high  stress,  subjects  face  a  specific  deadline  of  24  seconds.  In  addition,  they  must  complete 
a  secondary  communication  task  along  with  the  discrimination  task.  Discriminability  varies  the 
ease  with  which  enemy  and  friendly  submarines  can  be  discriminated  from  one  another  by  their 
pump  noise  signatures.  The  two  levels  are  easy  —  pump  signatures  are  two  standard  devia¬ 
tions  apart  (330  Hz  for  the  enemy  and  370  Hz  for  the  friendly),  and  difficult  —  pump  signa¬ 
tures  are  half  a  standard  deviation  apart  (360  Hz  for  the  enemy  and  370  Hz  for  the  friendly). 
Information  cost  manipulates  the  cost  incurred  when  asking  the  consultant  for  an  opinion  (C) 
relative  to  probing  (P).  The  three  levels  are:  C  <  P  —  cost  of  asking  the  consultant  is  less  than 
that  of  probing  for  a  new  measurement  (5  and  15,  respectively);  C  =  P  —  cost  of  asking  the 
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consultant  is  equal  to  the  cost  of  probing  (5  and  5,  respectively);  C  >  O  —  cost  of  asking  the 
consultant  is  greater  than  that  of  probing  (15  and  5,  respectively).  Consultant  expertise  is 
operationalized  in  terms  of  the  consultant's  measurement  error  relative  to  the  commander’s 
(subject's)  measurement  error.  The  two  levels  are  high  expertise  —  consultant's  measurement 
error  is  half  the  commander's  (i.e.,  half  the  measurement  standard  deviation)  and  equal  exper¬ 
tise  —  consultant's  and  commander's  measurement  errors  are  equal  (i.e.,  same  measurement 
standard  deviation). 

3.4  VARIABLES  MEASURED 

Several  dependent  variables  are  devised  to  record  the  timings  and  processes  of  infor¬ 
mation  gathering,  decisionmaking,  and  outcome.  The  information  gathering  and  decision¬ 
making  measures  are  associated  with  one  of  the  three  overt  decisions  commanders  are  called  on 
to  make.  The  first  decision  records  if  subjects  stopped  information  gathering  after  the  initial 
measurement  and  decided  if  the  detected  submarine  is  enemy  or  friendly,  or  elect  to  gather 
more  information.  The  data  yields  the  percentage  of  times  subjects  requested  additional  infor¬ 
mation.  The  second  decision  records  if  commanders  probe  for  a  new  measurement  or  ask  the 
opinion  of  the  consultant  The  percentage  of  times  the  subjects  probe  is  computed  from  these 
data.  The  third  decision  records  whether  the  commanders  decide  the  detected  submarine  is  an 
enemy  or  a  friendly.  These  latter  data  are  used  to  compute  the  percentage  of  times  the  com¬ 
manders  correctly  identified  the  submarine.  The  outcome  of  the  detection  task  is  evaluated  and 
a  feedback  score  generated.  This  reward  score  is  computed  in  the  following  way:  100  points 
minus  80  points  for  being  wrong  minus  information  costs.  Thus,  if  the  commander  correctly 
identified  the  submarine  and  no  additional  information  is  requested,  the  reward  score  is  100.  If 
the  commander  incorrectly  classified  the  submarine,  but  uses  no  additional  information,  the 
reward  score  is  20.  If  the  commander  is  wrong  and  incurs  an  (additional)  information  cost  of 
15,  then  the  reward  score  is  5.  After  each  discrimination  decision,  commanders  render  a  rating 
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as  to  their  confidence  in  their  decision.  These  ratings  are  made  on  a  seven-point  scale  where  1 
equals  very  low  and  7  equals  very  high  confidence  that  the  decision  about  the  detected  subma¬ 
rine  is  correct. 

Figure  3-1  shows  the  various  timing  measures  recorded.  All  intervals  are  measured  in 
seconds.  The  Five  measures  are: 

•  tfinal.  the  interval  from  the  onset  of  a  trial  to  the  end  of  a  trial. 

•  tri  (reaction  to  primary  task),  the  interval  from  the  onset  of  a  trial  to  the 
first  decision. 

•  tdi  (decision  for  primary  task),  the  interval  from  the  first  decision  to  the 
third  decision  (i.e.,  enemy  or  friendly). 

•  tf2  (reaction  to  secondary  task),  the  interval  from  the  onset  of  a  trial  until 
the  subject  reacts  to  the  secondary  task  by  sending  the  pump  noise 
measurement. 

•  td2  (decision  for  secondary  task),  the  interval  from  the  transmission  of  the 
secondary  pump  noise  measurement  until  the  subject  sends  the  verification 
of  the  measurement. 
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Figure  3-1.  Timeline  Showing  the  Five  Time  Intervals  Measured  and  Recorded. 
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3.5  WORKLOAD  ASSESSMENT 

To  assess  workload,  each  participant  completed  the  Subjective  Workload  Assessment 
Technique  or  SWAT  (Reid  et  al.,  1988)  scale  at  the  conclusion  of  each  experimental  condition. 
The  assessment  process  using  SWAT  involves  two  distinct  phases.  Phase  one  is  carried  out 
prior  to  data  collection.  Following  prescribed  procedures  (Reid  et  al.,  1981),  each  participant 
performs  a  card  sort  to  develop  a  unique  workload  scale  for  themselves.  Each  card  contains  a 
different  combination  of  the  three  workload  dimensions:  time  load,  mental  effort  load,  and 
psychological  stress  load.  In  turn,  each  workload  dimension  is  described  in  terms  of  three 
levels:  low,  moderate,  and  high.  Thus,  participants  sort  a  set  of  27  cards  (3  levels  of  time 
x  3  levels  of  mental  effort  x  3  levels  of  psychological  stress),  so  that  they  are  rank  ordered 
according  to  the  level  of  workload  described. 

The  second  phase  occurs  during  data  collection.  At  the  end  of  each  experimental  con¬ 
dition,  participants  rate  the  workload  they  just  experienced  according  to  the  three  dimensions  of 
time  load,  mental  effort  load,  and  psychological  stress  load.  Ratings  are  done  on  a  three-point 
scale  (low,  moderate,  high)  for  each  dimension.  Using  software  provided  by  Reid  et  al.,  each 
SWAT  score,  which  consists  of  one  number  from  each  of  the  three  dimensions  (e.g.,  1, 2, 2; 
representing  a  1  for  time  load,  a  2  for  mental  effort  load,  and  a  2  for  psychological  stress  load), 
is  converted  to  a  percent  workload  score  on  the  participant's  unique  scale.  Zero  percent  repre¬ 
sents  very  low  workload,  while  100  percent  represents  very  high  workload. 

3.6  EXPERIMENTAL  PARAMETERS 

The  values  needed  for  the  initial  pump  noise  measurement,  probe  measurement,  and 
measurement  for  input  into  the  consultant  algorithm  are  randomly  generated  from  a  Gaussian 
distribution  with  appropriate  mean  and  standard  deviation.  For  each  trial,  it  is  first  decided  if 
the  submarine  will  be  an  enemy  or  a  friendly  (ground  truth)  by  sampling  a  uniform  random 
distribution  of  zeros  and  ones  (i.e.,  probability  of  an  enemy  submarine  is  0.5).  If  the  subma¬ 
rine  is  to  be  an  enemy,  then  an  initial  pump  noise  value  is  drawn  from  the  enemy's  Gaussian 
distribution.  Depending  on  the  discriminability  condition,  this  distribution  has  a  mean  of 
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330  Hz  or  360  Hz  and  a  standard  deviation  of  20  Hz.  The  friendly  submarine  distribution 
always  has  a  mean  of  370  Hz  and  a  standard  deviation  of  20  Hz.  If  a  probe  measurement  is 
required,  it  is  drawn  from  the  same  distribution  as  the  initial  measurement 

Generation  of  the  measurement  used  by  the  consultant  is  done  in  a  similar  fashion.  If 
the  consultant  and  the  commander  are  equal  in  expertise,  then  the  input  value  to  the  consultant 
algorithm  is  randomly  generated  from  the  same  distribution  as  the  initial  measurement.  When 
the  consultant  is  a  better  expert  than  the  commander,  an  input  value  is  drawn  from  a  Gaussian 
distribution  with  a  standard  deviation  of  10  Hz  (and  mean  appropriate  to  the  submarine  type  — 
enemy  or  friendly). 

The  generation  of  all  measurement  values  is  done  only  once.  All  subjects  see  the  same 
measurements,  albeit  in  different  orderings  according  to  experimental  condition  and  replication. 

3.7  PROCEDURE 

Prior  the  onset  of  the  experiment,  all  subjects  first  read  "Instructions  for  the  TOSCA-II 
Simulation  Game"  (a  copy  of  these  instructions  are  in  Appendix  A).  These  instructions  pro¬ 
vide  an  overview  of  the  task,  describe  the  subject’s  role,  and  provide  a  motivational  induction 
to  take  the  simulation  game  seriously,  try  hard,  and  do  well.  Subjects  next  read  "Instructions 
for  the  TOSCA-II  Simulator"  (a  copy  of  these  instructions  are  also  in  Appendix  A).  The 
second  set  of  instructions  are  designed  to  tell  the  subjects  in  specific  terms  what  to  do  and  how 
to  do  it.  Each  area  of  the  display  screen  is  described  and  its  function  explained.  Feedback, 
scoring,  timing,  and  the  secondary  task  are  also  explained.  Lastly,  subjects  are  reminded  to 
complete  the  SWAT  scale  at  the  end  of  each  experimental  condition. 

Subjects  work  at  their  own  Apple  Macintosh  microcomputer  and  make  all  entries  via  the 
mouse  (or,  if  they  choose,  a  track  ball).  For  a  low-stress  trial,  the  display  shown  in  Fig.  3-2 
appears,  while  for  a  high-stress  trial,  Fig.  3-3  shows  the  display.  For  low  stress,  a  trial  begins 
when  an  initial  measurement  appears  on  the  screen  (e.g.,  a  pump  noise  signature  of  382  Hz). 
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The  commanders  indicate  if  they  think  the  detected  submarine  is  an  enemy  or  a  friendly,  or 
more  information  is  desired  (first  decision ).  If  more  information  is  requested,  the  commanders 
indicate  if  they  desire  a  probe  to  obtain  another  measurement  or  the  opinion  of  the  consultant 
(second  decision).  At  all  times,  the  screen  clearly  displays  the  cost  of  each  course  of  action,  as 
well  as  the  consultant's  expertise  vis-a-vis  the  commander.  After  a  determination  of  enemy  or 
friendly  (third  decision),  subjects  enter  their  confidence  rating  and  then  request  feedback  as  to 
the  correctness  of  their  decision.  The  procedure  is  similar  for  high-stress  trials  except  the 
subjects  must  also  complete  the  secondary  task.  This  entails  noting  the  secondary  pump  noise 
measurement,  sending  the  measurement,  and  then  (ten  seconds  later)  verifying  that  the  pump 
noise  measurement  returned  is  in  fact  the  one  that  was  sent.  Feedback  in  these  cases  also 
includes  whether  the  commander's  verification  action  was  correct. 

At  the  end  of  an  experimental  condition,  a  message  appears  on  the  display  screen 
directing  subjects  to  complete  a  SWAT  scale  for  that  condition.  In  all  24  SWAT  scales  are 
completed. 

3.8  DESIGN 

A  complete  crossing  of  the  four  independent  variables  —  stress,  information  cost, 
consultant  expertise,  and  discriminability  —  yields  24  experimental  conditions  (i.e.,  2x3 
x  2  x  2).  The  design  is  completely  within-subjects,  in  that  each  subject  experiences  all  24 
conditions  in  a  partially  counterbalanced,  partially  random  ordering.  The  experimental  condi¬ 
tions  are  organized  by  the  stress  factors  into  four  blocks  (two  low  and  two  high)  and  presented 
to  the  subjects  in  an  ABBA  or  BAAB  counterbalanced  ordering.  Within  each  stress  block, 
discriminability  is  also  ordered  in  ABBA  or  BAAB  counterbalancing.  The  remaining  factors 
are  randomized  within  each  block.  Each  experimental  condition  is  replicated  15  times  for  a 
total  of  360  trials  for  a  civilian  subject  and  7  times  for  a  total  of  105  trials  for  a  military  subject. 
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SECTION  4 

EXPERIMENT  RESULTS 

4.1  MANIPULATION  CHECKS 

4.1.1  The  Stress  Induction 

A  primary  goal  of  the  experiment  is  to  observe  decisionmaking  under  stress.  Thus, 
it  is  important  to  confirm  that  a  condition  relatively  high  in  cognitive  stress  was  produced. 

To  check  the  veracity  of  the  high-stress  manipulation,  we  turn  to  the  workload  ratings.  It  is 
expected  that  a  successful  stress  manipulation  will  produce  higher  workload  ratings  in  the 
high-stress  rather  than  low-stress  condition.  In  the  civilian  sample,  subjects  reported  a  mean 
workload  score  of  38  percent  under  high  stress  and  a  score  of  25  percent  under  low  stress. 

The  difference  in  the  expected  direction  is  significant  at  p  <  .04  (one-tail).  Similarly,  military 
decisionmakers  reported  a  mean  workload  score  of  32  percent  under  high  stress  and  a  score  of 
17  percent  under  low  stress.  In  this  case,  the  difference  is  significant  (p  <  .06,  one-tail)  in  the 
expected  direction.  It  appears  safe  to  say  that  the  stress  manipulation  had  a  significant  effect 
and  the  effect  is  as  expected. 

4.1.2  The  Discrimination  Manipulation 

Using  similar  procedures,  it  is  also  possible  to  examine  the  manipulation  effectiveness 
for  the  discriminability  factor.  If  successfully  manipulated,  this  factor  will  produce  higher 
workload  ratings  in  the  difficult  versus  the  easy-discrimination  condition.  As  anticipated, 
subjects  in  the  civilian  sample  reported  significantly  higher  mean  workload  ratings  in  the 
difficult  than  easy-discrimination  condition  (means  equal  43  percent  and  21  percent,  respec¬ 
tively;  p  <  .004).  A  weaker,  but  essentially  similar,  pattern  of  results  is  found  for  the  military 
sample  (means  equal  27  percent  for  difficult  and  22  percent  for  easy  discrimination  conditions; 
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p  <  .07,  one-tail).  We  conclude  that  the  discrimination  manipulation  produced  an  easy  and 
difficult  condition  as  desired. 


4.1.3  A  Confirmation  of  Findings 
INFORMATION-SEEKING  STRATEGIES 

The  analyses  to  be  described  first  compare  the  information-seeking  strategies  observed 
in  the  present  experiment  (which  we  will  call  TOSCA-II)  to  those  observed  by  Kastner  et  al.  - 
(1989)  in  the  first  TOSCA  experiment  (to  be  referred  to  as  TOSCA-I).  The  comparison  is 
performed  with  data  collected  for  the  civilian  sample  in  low  stress.  These  are  the  conditions 
most  comparable  to  those  of  TOSCA-I. 

As  we  can  see  from  Fig.  4-1  in  the  easy -discrimination  condition,  decisionmakers  in 
TOSCA-II  exhibit  a  weaker  propensity  to  make  a  decision  as  to  the  hostility  of  the  detected 
submarine  without  requesting  additional  information  than  decisionmakers  in  TOSCA-I.  The 
pattern  of  results  for  TOSCA-I  and  II,  seen  in  the  difficult  discrimination  condition,  is  also 
similar.  For  TOSCA-I,  decisionmakers  ask  for  more  information  twice  as  often  as  not  seeking 
more  data  while  for  TOSCA-II,  the  ratio  is  5-to-2  in  favor  of  requesting  more  information. 
Thus,  the  initial  patterns  of  information  seeking  appear  quite  similar  for  the  two  TOSCA 
experiments. 
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The  effects  of  cost  on  information  seeking  in  TOSCA-I  are  presented  in  Fig.  4-2a. 

The  comparable  findings  from  TOSCA-II  are  shown  in  Fig.  4-2b.  Note  that  the  general  pat¬ 
terns  of  results,  for  the  conditions  where  probing  was  less  expensive  or  more  expensive  than 
consulting,  are  similar  across  experiments.  When  probing  is  less  expensive,  a  majority  of 
decisions  favor  probing.  When  consulting  is  less  expensive,  a  majority  of  decisions  favor 
consulting.  Moreover,  this  is  true  regardless  of  the  discrimination  condition.  The  pattern  of 
results  ceases  to  be  similar  when  the  condition  of  equal  cost  is  examined.  In  TOSCA-I,  deci¬ 
sionmakers  appeared  to  be  indifferent  or  show  a  weak  tendency  toward  consulting,  when  the 
costs  of  probing  and  consulting  are  equal;  while  in  TOSCA-II,  participants  show  a  definite 
preference  for  probing  in  both  discriminability  conditions.  In  the  easy  condition,  the  ratio  is 
3-to-2;  while  in  the  difficult  case,  the  ratio  is  2-to-l,  both  in  favor  of  probing.  That  such  a  bias 
would  exist,  when  the  consultant  is  equal  in  expertise  and  consulting  costs  the  same  as  prob¬ 
ing,  was  hypothesized  in  TOSCA-I.  It  was  argued  that  decisionmakers  would  favor  the  raw 
data  of  the  probe  over  the  processed  information  provided  by  the  consultant,  because  there  are 
more  "bits"  of  available  information  in  a  raw  measurement.  This  hypothesis  was  confirmed  by 
the  results  of  TOSCA-I.  Perhaps  in  the  current  experiment,  this  tendency  is  more  strongly 
expressed.  It  was  also  hypothesized  (Kastner  et  al.,  1989)  that  some  motivation  may  exist  for 
participants  to  strive  to  maintain  as  much  control  over  the  decision  process  as  possible  (see 
Trope,  1975,  and  Trope  and  Brickman,  1975).  These  issues  will  be  taken  up  subsequently. 

TOSCA-I  results  confirmed  expectations  that  decisionmakers  would  turn  to  the  consul¬ 
tant  for  help  more  frequently  when  the  consultant  is  superior  in  expertise,  and  that  when  no 
difference  in  expertise  exists,  decisionmakers  would  favor  probing.  Figures  4-3a  and  4-3b 
show  that  TOSCA-II  findings  strongly  corroborate  these  results.  In  the  easy  discrimination 
conditions,  the  results  are  identical  between  experiments,  while  in  the  difficult  discrimination 
case,  the  pattern  of  results  is  similar. 
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Figure  4-2.  Decisions  to  Probe  or  Consult  as  a  Function  of  Cost  and 
Discriminability  for  TOSCA-I(a)  and  TOSCA-II(b). 
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Figure  4-3.  Decision  to  Probe  and  Consult  as  Function  of  Consultant  Expertise 
and  Discriminability  for  TOSCA-I(a)  and  TOSCA-II(b). 


The  next  comparison  examines  information-seeking  strategies  as  affected  by  the  inde¬ 
pendent  variables  cost,  expertise,  and  discriminability.  Figure  4-4a  presents  the  results 
obtained  from  TOSCA-I  and  Fig.  4-4b  shows  the  current  findings  of  TOSCA-II.  In  the  easy 
discrimination  condition,  the  pattern  of  results  is  quite  similar,  with  one  exception.  When  the 
consultant  and  decisionmaker  are  equal  in  expertise  and  the  consultant  is  less  expensive, 
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Figure  4-4b.  Decision  to  Probe  or  Consult  as  a  Function  of  Cost,  Expertise, 
and  Discriminability  for  TOSCA-D. 
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subjects  show  ns  preference  for  probing  over  consulting  in  TOSCA-II  while  in  TOSCA-I, 
decisionmakers  selected  consulting  by  a  2-to-l  margin.  Results  under  difficult  discrimination 
show  a  reasonably  good  match  between  the  two  experiments.  The  one  reversal  occurs  when 
the  probe  is  less  expensive  and  the  consultant  is  a  better  expert.  In  this  condition,  TOSCA-II 
decisionmakers  choose  probing  3-to-2  over  consulting,  while  TOSCA-I  decisionmakers 
slightly  favor  consulting.  Another  mismatch,  but  not  a  reversal,  occurs  when  consulting  is 
less  expensive  and  no  difference  in  expertise  exists.  The  results  from  TOSCA-I  show  a  clear 
preference  for  consulting,  but  TOSCA-II  decisionmakers  are  indifferent  about  probing  or 
consulting  in  this  condition.  It  is  interesting  to  note  that  all  the  mismatches  are  due  to  a  greater 
propensity  for  TOSCA-II  decisionmakers  to  probe  than  consult.  A  discussion  of  this  issue  will 
be  taken  up  later. 

It  was  hypothesized  for  TOSCA-I  that  decisionmakers  would  choose  between  probing 
and  consulting  according  to  the  following  cost-benefit  arguments: 

•  If  decisionmaker  and  consultant  are  equal  experts,  decisionmakers  will 
select  the  least  costly  information  source. 

•  If  the  costs  of  probing  and  consulting  are  equal,  then  decisionmakers  will 
seek  information  from  the  (superior)  consultant. 

•  If  cost  and  expertise  are  both  equal,  then  decisionmakers  will  probe, 
because  a  three-digit  measurement  provides  more  bits  of  information  than 
a  single  opinion. 

The  obtained  results  confirmed  all  predictions  and  it  was  concluded  that  decisionmakers  per¬ 
form  some  kind  of  information  cost-benefit  analysis  following  the  above  three  rules  to  decide 
what  information  source  to  choose. 

For  comparable  conditions,  it  is  expected  that  decisionmakers  in  the  current  experiment 
will  follow  the  same  cost-benefit  decision  rules.  Table  4-1  shows  the  outcomes  for  both 
TOSCA-I  and  TOSCA-II.  It  appears  that  TOSCA-II  decisionmakers  follow  the  same  cost- 
benefit  decision  rules  as  decisionmakers  in  TOSCA-I.  The  only  exception  occurs  when  the 
consultant  and  decisionmakers  are  equal  in  expertise  and  the  consultant  is  cheaper.  The  cost- 
benefit  decision  rules  imply  consulting  and  TOSCA-I  decisionmakers  followed  suit,  but 
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TABLE  4-1.  PREDICTED  AND  OBTAINED  RESULTS  FROM  TOSCA-I  AND  TOSCA-E 
FOR  THE  INFORMATION  COST-BENEFIT  DECISION  RULES 
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*  Essentially  it  is  not  possible  from  the  cost-benefit  decision  rules  to  determine  if  cost  or 
expertise  will  prevail. 


TOSCA-II  participants  select  probing  or  show  no  preference.  This  is  another  demonstration 
that  TOSCA-n  subjects  show  a  marked  tendency  to  probe  over  consult.  As  stated  above,  this 
issue  will  be  discussed  later. 


DECISION  PERFORMANCE 

TOSCA-I  reported  two  significant  findings  nested  within  the  two  discriminability 
conditions,  one  dealing  with  information  cost  and  the  other  dealing  with  consultant  expertise. 

A  comparison  of  performance  between  the  two  studies  for  information  cost  is  presented  in 
Fig.  4-5.  The  pattern  of  results  obtained  for  TOSCA-I  and  TOSCA-II  appears  quite  similar. 
The  most  noticeable  difference  occurs  when  the  probe  is  least  costly.  In  this  case,  performance 
of  the  TOSCA-n  subjects  falls  ten  percentage  points  below  these  reported  for  TOSCA-I. 

These  differences,  however,  coupled  with  other  small  differences  do  nai  lead  to  the  same 
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Figure  4-5.  Performance  as  a  Function  of  Cost  and  Discrimination  for 
TOSCA-I  and  TOSCA-II. 

conclusions.  That  is,  based  on  TOSCA-I  findings  it  was  concluded  that  because  the  deci¬ 
sionmakers  consulted  more  when  the  probe  was  most  costly  than  in  the  other  two  cost  cases, 
decisionmakers  must  not  be  getting  as  much  information  as  they  need  from  the  consultant  to 
make  good  discriminations.  In  TOSCA-II,  the  probing  and  consulting  strategies  across  cost 
conditions  appear  the  same,  but  decisionmakers  do  not  seem  to  use  the  richer  probe  informa¬ 
tion  as  effectively  as  their  counterparts  in  TOSCA-I. 

The  results  reported  by  each  experiment  for  consultant  expertise  are  compared  in 
Fig.  4-6.  Once  again,  the  pattern  of  results  seems  quite  similar.  However,  the  significant 
difference  in  performance  found  for  TOSCA-I,  favoring  the  consultant  when  more  expert 
over  the  equal  expert  consultant  when  in  the  easy  discriminability  condition,  is  not  found  in 
TOSCA-II.  The  means  are  in  the  same  direction,  but  the  difference  is  not  significant.  In  the 
difficult  discrimination  condition,  the  performance  of  the  two  expertise  conditions  in  TOSCA-II 
arc  the  reverse  of  those  observed  for  TOSCA-I.  That  is,  in  TOSCA-II  the  performance  in  the 
equal  expertise  condition  is  superior  to  that  of  the  consultant-more-expert  condition  while  the 
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Figure  4-6.  Performance  as  a  Function  of  Expertise  and  Discriminability  for 
TOSCA-I  and  TOSCA-II. 

performance  of  this  latter  expertise  condition  was  superior  in  TOSCA-I.  Decisionmakers  do 
take  advantage  of  the  better  consultant  in  the  more  expert  consultant  condition  and  consult 
more;  but,  the  consultant  still  faces  a  difficult  discrimination  (as  do  the  decisionmakers)  and  his 
advice  may  not  be  as  good  as  the  richer  probe  information  enjoyed  in  the  equal-expertise 
condition,  where  the  favored  strategy  is  probing. 

TOSCA-I  also  reported  significant  cost-by-expertise  interactions  that  are  shown  in 
Fig.  4-7a.  For  comparison,  the  performance  of  these  conditions  for  TOSCA-II  are  depicted  in 
Fig.  4-7b.  One  might  construe  that  the  patterns  of  results  for  the  easy-discrimination  condition 
are  vaguely  similar  between  the  two  experiments.  Both  show  that  the  consultant-more-expert 
condition  achieves  better  performance  scores  than  the  equal-expertise  condition  where  probing 
is  less  or  more  expensive  than  consulting.  In  contrast  to  TOSCA-I  results,  TOSCA-II  findings 
show  that  performance  in  the  equal-cost  condition  is  superior  for  the  equal  expertise  case. 
Result  patterns  for  the  two  experiments  appear  quite  different  in  the  difficult-discrimination 
condition.  The  most  obvious  difference  occurs  where  probing  is  more  costly  than  consulting. 
For  TOSCA-I,  the  performance  of  the  two  expertise  levels  were  the  same,  while  in  TOSCA-II, 
the  equal-expertise  case  shows  considerably  better  performance  than  the  consultant-more- 
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expert  case.  Once  again,  it  seems  that  consultants  in  difficult-discrimination  conditions  cannot 
provide  as  good  information  as  a  probe. 


Figure  4-7.  Performance  as  a  Function  of  Cost,  Expertise,  and  Discriminability 
for  TOSCA-I(a)  AND  TOSCA-D(b). 
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SUMMARY 

The  information-seeking  strategies  of  probing  and  consulting  are  quite  comparable 
between  experiments.  Although  a  few  anomalies  exist,  by-and-large  the  information-seeking 
strategies  observed  for  TOSCA-I  are  corroborated  by  the  TOSCA-II  findings.  This  further 
reinforces  the  argument  that  decisionmakers  apply  a  kind  of  cost-benefit  analysis  when  decid¬ 
ing  among  the  different  information-seeking  strategies. 

The  picture  for  performance  is  different.  While  some  of  the  patterns  look  similar,  there 
are  enough  differences  to  lead  to  different  conclusions  in  TOSCA-II.  TOSCA-II  decision¬ 
makers  seem  less  able  to  employ  all  the  information  available  in  a  probe's  measurement  and 
suffer  more  from  the  consultant's  poorer  performance  in  difficult  discrimination  conditions. 

4.2  ANALYSIS  OF  DECISIONMAKING  PERFORMANCE  AND  INFORMATION¬ 
SEEKING  STRATEGIES  FOR  THE  CIVILIAN  SAMPLE 

4.2.1  Information  Seeking 

Part  of  the  discussion  examining  information-seeking  strategies  appears  in  the  previous 
subsection  (4.1.3),  where  such  strategy  patterns  are  compared  for  TOSCA-I  and  TOSCA-II. 

In  this  section  we  will  look  more  deeply,  examining  the  effects  of  stress  as  well  as  the  other 
independent  variables.  To  best  gauge  the  effects  of  stress,  the  remaining  three  independent 
variables  are  analyzed  within  conditions  of  high  and  low  stress. 

In  situations  of  low  stress,  decisionmakers,  as  shown  in  Fig.  4-8,  requested  informa¬ 
tion  significantly  more  often  in  the  probe-less-costly  and  equal  cost  conditions  and  requested 
the  least  amount  of  information  in  the  probe-more-costly  condition  (p  <  .01).  High  stress 
apparently  induces  more  noise  (error)  into  the  situation,  as  the  patterns  of  information  requests 
are  similar,  but  do  not  reach  acceptable  significance  levels. 

It  is  interesting  to  note  that  across  all  conditions,  decisionmakers  request  information 
more  than  54  percent  of  the  time  and  they  request  information  equally  in  low-  and  high-stress 
situations.  When  the  probe  is  more  costly  than  the  consultant,  however,  decisionmakers  seem 
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Figure  4-8.  Percent  Information  Requests  as  a  Function  of  Cost  and  Stress 
for  the  Civilian  Sample. 

a  bit  more  reluctant  to  ask  for  additional  information.  It  is  as  if  they  have  some  reluctance  to 
ask  the  opinion  of  the  consultant,  even  when  the  consultant  is  cheaper. 

This  aversion  to  ask  the  consultant  seems  evident  in  the  expertise  findings.  There  is  no 
increase  in  tendency  for  decisionmakers  to  request  more  information  in  conditions  where  the 
consultant  is  a  better  expert  compared  to  the  equal  expertise  condition;  and  this  is  true  under 
low  and  high  stress.  In  fact,  there  is  no  evidence  that  decisionmakers  make  more  information 
requests  in  cases  where  the  consultant  is  cheaper  than  the  probe  and  a  better  expert  than  the 
decisionmakers.  Information  requests  are  flat  across  the  various  joint  conditions  of  cost  and 
expertise,  regardless  of  stress  conditions.  One  would  expect  that  higher  expertise  and  a 
cheaper  consultant  would  entice  people  to  ask  for  more  information  than  in  conditions  that  lack 
one  of  these  assets.  This  does  not  seem  to  be  the  case.  We  will  be  in  a  better  position  to  dis¬ 
cuss  reasons  for  this  behavior  when  the  patterns  of  probing  and  consulting  are  addressed. 

Figure  4-9  shows  information  requests  for  the  easy-  and  difficult-discrimination  condi¬ 
tions  in  low  and  high  stress.  Under  low  stress,  decisionmakers  request  significantly  more 
information  in  difficult-  than  easy-discrimination  conditions  (j2  <  .02).  The  addition  of  high 
stress  further  increases  the  desire  to  seek  information  under  difficult-  than  easy-discrimination 
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Figure  4-9.  Percent  Information  Requests  as  a  Function  of  Discriminability 
and  Level .  f  Stress  for  Civilians. 

conditions  (p  <  .001).  The  results  reported  earlier  in  TOSCA-I  that  decisionmakers  seek  more 
information  in  difficult-  than  easy-discrimination  conditions  is  supported,  and  it  appears  that 
stress  further  increases  the  desire  for  information. 

Discriminability  and  cost  also  play  a  role  in  information  requests,  but  only  in  high 
stress.  Figure  4-10  shows  that  the  pattern  for  the  discriminability-by-cost  interaction  under 
low  stress  is  similar  to  the  pattern  under  high  stress,  but  not  as  extreme.  Essentially,  under 
high  stress  the  difference  between  easy-  and  difficult-discrimination  conditions,  when  the 
probe  is  less  expensive,  is  significantly  greater  than  the  same  contrast  when  the  probe  is  more 
expensive  (p  <  .05).  In  high-stress  difficult-discrimination  situations,  decisionmakers  request 
additional  information  75  to  80  percent  of  the  time  when  the  probe  is  cheaper  or  equal  in  cost  to 
consulting.  However,  when  the  probe  is  more  expensive  than  the  consultant,  information 
requests  drop  sharply  in  the  difficult-discrimination  condition.  This  is  another  indication  that 
decisionmakers  avoid  asking  for  information  when  cost  conditions  favor  the  consultant. 
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Figure  4-10.  Percent  Information  Requests  as  a  Function  of  Cost, 
Discriminability,  and  Level  of  Stress  for  Civilians. 


4.2.2  Probing  and  Consulting  Strategies 

Figure  4-11  depicts  two  important  results.  First,  decisionmakers  probe  a  majority  of 
the  times  they  request  information  (i.e.,  more  than  58  percent  of  the  time).  Secondly,  results 
also  show  that  participants  probe  significantly  more  in  low-  than  in  high-stress  situations 
(p  <  .05).  Apparently,  decisionmakers  turn  away  from  the  probe  and  increase  consulting  in 
stressful  situations.  Under  such  conditions  they  may  feel  the  consultant's  processed  opinion  is 
easier  than  a  raw  measurement  to  incorporate  into  their  current  position. 

The  probing  and  consulting  strategies  for  the  three  cost  conditions  vary  significantly 
in  low-  and  in  high-stress  situations  (p  <  .001,  respectively).  Moreover,  the  same  pattern  of 
probing  and  consulting  is  seen  in  low  and  in  high  stress.  Decisionmakers  probe  extensively 
when  it  is  lowest  in  cost  (83  percent  of  the  time)  and  consult  most  when  the  consultant  is 
cheapest  (70  percent  of  the  time).  In  the  equal-cost  condition,  participants  favor  probing  62 
percent  of  the  time.  As  reported  in  the  TOSCA-I  experiment,  participants  are  sensitive  to  the 
cost  of  information  and,  other  things  being  equal,  turn  to  the  source  that  is  the  cheapest.  In  the 
equal-cost  condition,  however,  decisionmakers  are  not  indifferent  as  they  were  in  TOSCA-I; 
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Figure  4-11.  Information  Source  Used  Under  High  and  Low  Conditions  of 
Stress  by  Civilian  Sample. 

instead  they  favor  probing  62  percent  of  the  time.  The  pattern  of  probing  and  consulting 
observed  for  the  cost  condition  is,  however,  unchanged  by  stress. 

Similarly,  the  probing  and  consulting  strategies  observed  in  the  expertise  conditions 
are  also  unaffected  by  stress.  In  low-  and  high-stress  situations,  decisionmakers  consulted  a 
majority  of  the  times  when  the  consultant  is  the  superior  expert,  and  probe  a  majority  of  the 
time  when  the  consultant  and  decisionmaker  are  equal  in  expertise  (p  <  .002,  respectively). 
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The  tendency  for  decisionmakers  to  turn  to  the  consultant  in  adverse  conditions  is  fur¬ 
ther  evident  in  Fig.  4-12.  Under  low  stress,  participants  probe  for  their  additional  information 
about  61  percent  of  the  time  in  easy-  and  difficult-  discrimination  conditions.  In  high-stress 
situations,  the  tendency  to  probe  falls  about  17  percent  in  the  difficult-discrimination  versus 
easy-discrimination  condition  (p  ^  .05).  Clearly,  decisionmakers  are  turning  away  from 
probing  in  conditions  of  high-stress  and  difficult  task.  As  we  have  suggested  before,  we 
believe  participants  are  turning  to  the  easier-to-assimilate  consultant  opinion  when  the  situation 
they  are  in  becomes  cognitively  loaded. 
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Figure  4- 1 2.  Percent  Probing  as  a  Function  of  Discriminability  and  Level  of 
Stress  for  Civilian  Samples. 


Probing  and  consulting  strategies  are  affected  by  the  joint  effect  of  cost  and  expertise; 
but,  as  Fig.  4-13  shows,  the  effect  of  stress  on  this  interaction  is  subtle.  At  first,  it  may  appear 
that  the  patterns  seen  for  high  and  low  stress  are  alike.  Careful  inspection,  however,  reveals 
some  interesting  differences.  Examine  the  difference  in  probing  between  the  equal  expertise 
and  consultant-better-expert  condition  when  probing  is  less  expensive  and  stress  is  low. 
Probing  decreases  when  the  consultant  is  a  better  expert  by  about  25  percentage  points.  A  45 
percentage  point  drop  in  probing  occurs  when  the  consultant  is  a  better  expert  and  also  cheaper 
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Figure  4-13.  Percent  Probing  as  a  Function  of  Cost,  Expertise,  and  Level  of 
Stress  for  Civilians. 


than  the  probe.  Remember  a  drop  in  probing  means  a  commensurate  increase  in  consulting. 
These  differences  in  probing  and  consulting  are  as  expected.  Being  able  to  turn  to  a  better 
(if  expensive)  expert  may  allure  decisionmakers  some  of  the  time,  and  turning  to  a  better  and 
less  expensive  expert  is  even  more  alluring.  High  stress,  however,  appears  to  change  this 
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tendency.  When  the  probe  is  less  expensive,  a  41  percentage  point  drop  in  probing  is  observed 
when  the  consultant  is  a  better  expert,  while  only  a  28  percentage  point  decline  is  seen  when 
the  consultant  is  cheaper.  When  experiencing  high  stress,  decisionmakers  seem  to  place  a 
higher  premium  on  the  consultant's  being  expert  than  cheap. 

4.2.3  Decision  Performance 

In  this  subsection  we  examine  how  well  decisionmakers  perform  the  discrimination 
task  under  stress.  The  performance  variable  in  this  case  is  the  reward  score.  Recall  that  this 
score  reflects  correctness  as  well  as  the  cost  of  information  requests. 

Analysis  of  performance  brings  to  light  an  unexpected  finding:  decisionmakers  appear 
to  perform  better  under  stress.  In  high-stress,  the  reward  score  is  77.5  while  in  low  stress, 
the  reward  score  is  74.5  (p  <  .001).  This  small  but  significant  difference  occurs,  we  believe, 
because  stress  tends  to  arouse  participants  and  make  them  more  vigilant  (see  Janis  and  Mann, 
1977).  This  increase  in  arousal,  other  things  being  equal,  motivates  participants  to  work  a  little 
harder  at  the  discrimination  task  resulting  in  a  better  performance  score  (Yerkes  and  Dodson, 
1908). 

Cost  of  information  appears  to  significantly  impact  performance  (ps  <  .002);  however, 
the  pattern  of  results  is  quite  similar  under  high-  and  low-stress  conditions.  Essentially, 
decisionmakers  perform  best  in  the  equal-cost  condition,  while  performance  in  the  probe-more- 
costly  and  least-costly  conditions  is  about  the  same.  The  ratio  of  probing  to  consulting  in  the 
equal-cost  case  is  more  balanced  compared  to  the  other  cost  conditions.  Perhaps  this  particular 
blend  of  information  sources  yields  the  best  of  the  probe's  richer  information  and  the  easier-to- 
incorporate  consultant's  opinion. 

Figure  4-14  shows  the  participant's  performance,  by  expertise  level  for  low-  and  high- 
stress  situations.  When  stress  is  low,  participants  achieve  best  in  the  equal-expertise  condition 
(p  <  .001).  The  obverse  is  true  under  high  stress,  where  participants  do  best  when  the  con¬ 
sultant  is  a  better  expert  (p  <  .001).  Recall  that  decisionmakers  probe  substantially  in  the 
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Figure  4-14.  Reward  Score  as  a  Function  of  Expertise  under  Low-  and  High- 
Stress  Levels  for  Civilian  Sample. 


equal-cost  condition  and  consult  more  when  the  consultant  is  a  better  expert.  We  speculate 
that,  under  low  stress,  decisionmakers  are  less  cognitively  loaded  and  have  more  time  (or  cog¬ 
nitive  capacity)  to  use  the  information-rich  probe.  They  have  the  opportunity  to  interpret  the 
information  and  fold  it  into  their  current  belief  about  the  discrimination  task.  However,  when 
the  situation  is  stressful,  less  time  (or  cognitive  capacity)  is  available  to  deal  with  the  probe; 
thus  the  available  information  is  not  effectively  used,  and  decisionmakers  do  better  with  the 
processed  opinions  of  the  consultant. 

The  performance  levels  of  decisionmakers  are  significantly  better  when  the  discrimi¬ 
nation  task  is  easy  than  difficult  (ps  <  .007)  and  this  difference  seems  unaffected  by  stress. 
Closer  scrutiny  reveals  that  while  performance  is  the  same  for  difficult  discriminations  (a  score 
of  64  in  both  low  and  high  stress),  participants  performed  significantly  better  under  high  stress 
(scores  of  90  versus  85  for  low  and  high  stress,  respectively)  when  the  discrimination  task  is 
easy  (p  <,  .05).  We  speculated  that  the  better  overall  performance  reported  earlier  for  the  high- 
stress  condition  is  due  to  an  increase  in  arousal  or  motivation.  It  now  appears  that  increases  in 
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arousal  or  motivation  foster  better  performance  only  when  the  discrimination  task  is  easy  and 
not  when  the  task  is  difficult. 

Figure  4-15  shows  the  combined  effects  of  cost  and  expertise  on  reward  score  for  the 
low-stress  level.  Little  performance  difference  is  observed  among  the  three  cost  levels  when 
in  the  consultant-a-better-expert  condition.  In  the  equal-expertise  condition,  however,  perfor¬ 
mance  varies  over  the  cost  levels  with  the  best  performance  occurring  at  the  equal-cost  level. 
These  differences  are  not  observed  under  high  levels  of  stress. 
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Figure  4-15.  Reward  Score  as  a  function  of  Cost  and  Expertise  under  Low 
Stress  for  Civilian  Sample. 

Inspection  of  the  probing  and  consulting  patterns,  as  well  as  the  workload  data,  sheds 
no  light  on  why  the  equal-cost  condition  should  perform  better  than  the  other  two  cost  levels  in 
the  equal-expertise  situation.  Participants  probe  heavily  in  the  equal-cost  condition  (98  percent), 
but  also  do  so  when  the  probe  is  least  expensive  (99  percent),  yet  these  two  treatments  show 
quite  different  performance  outcomes.  Reported  workloads  for  these  treatments  are  also  about 
the  same.  Only  further  experimentation  will  resolve  this  issue. 

There  appears  to  be  no  significant  joint  effects  of  cost  and  discriminability  when  stress 
is  low,  but  a  significant  interaction  is  observed  when  stress  level  is  high  (p  <  .004).  Figure 
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4-16  shows  that  the  performance  levels  for  the  three  cost  conditions  are  almost  the  same  when 
discrimination  is  easy,  but  considerable  differences  in  performance  among  the  cost  levels  is 
seen  when  the  discrimination  condition  is  difficult  In  this  situation,  performance  is  best .  le 
equal-cost  condition,  while  the  performance  levels  for  the  other  two  cost  treatments  are  about 
equal.  Results  discussed  earlier  showed  that  civilians  probed  predominately  in  the  probe-least- 
expensive  condition  and  consulted  predominately  in  the  consultant-least-expensive  condition;  . 
however,  they  probed  and  consulted  about  equally  in  the  equal-cost  condition.  In  this  high- 
stress  difficult-discrimination  situation  we  surmise  that  this  balance  of  probing  and  consulting 
provided  the  decisionmakers  with  the  best  blend  of  easy-to-use  information  from  the  consultant 
and  rich  information  from  the  probes,  leading  to  better  performance. 
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Figure  4-16.  Reward  Score  as  a  Function  of  Cost  and  Discriminability  under 
High  Stress  for  Civilian  Sample. 


The  joint  effect  of  expertise  and  discriminability  impacts  performance  in  both  low  and 
high  levels  of  stress.  In  low  stress  (see  Fig.  4-17),  performance  at  both  expertise  levels  is 
about  the  same  under  easy  discrimination,  while  in  difficult  discrimination  performance  is  best 
when  the  consultant  and  decisionmaker  are  equal  in  expertise  as  compared  to  the  level  where 
the  consultant  is  a  superior  expert  (p  <  .001).  Figure  4-17  also  shows  that  under  high  stress 
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Figure  4-17.  Reward  Score  as  a  Function  of  Expertise,  Discriminability,  and 
Level  of  Stress  for  Civilian  Sample. 


the  difference  between  the  two  expertise  conditions  is  twice  as  large  in  the  difficult-  versus 
easy-discrimination  situation  (difference  of  4.8  versus  9.7,  respectively,  p  <  .06).  Moreover, 
in  contrast  to  the  difference  seen  in  low  stress,  under  high  stress  the  consultant-a-better-expert 
condition  performs  better  than  the  equal-expertise  condition,  when  discriminations  are  difficult 
Looking  back,  we  see  that  decisionmakers  probed  heavily  in  the  equal-expertise,  difficult- 
discrimination  treatment.  Under  low  stress,  decisionmakers  have  the  time  to  extract  the 
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information  available  in  the  probe's  measurement.  Under  high  stress,  participants  heavily 
consulted  in  the  consultant-a-superior-expert  condition,  when  discriminations  are  difficult  In 
this  time-critical  situation,  subjects  do  better  when  they  can  use  the  processed  information  of 
the  superior  expert. 

4.2.4  Workload 

The  analysis  of  subjective  workload  parallels  the  analysis  of  performance.  That  is,  the 
effects  of  the  various  independent  variables  on  subjective  workload  scores  are  observed  nested 
within  low-  and  high-stress  conditions. 

The  effect  of  information  cost  on  workload  for  both  the  high-  and  low-stress  conditions 
is  presented  in  Fig.  4-18.  The  three  cost  treatments  appear  to  have  no  affect  on  workload  when 
stress  is  low,  while  a  significant  effect  of  cost  is  observed  under  high  stress  (p<  .02).  In  con¬ 
ditions  of  high  stress,  participants  report  low  workload  when  the  probe  and  consultant  are 
equal  in  cost  It  was  noted  earlier  that  participants  exhibited  higher  performance  in  the  equal 
cost  condition  than  the  other  two  cost  levels.  This  performance  result  was  explained  by  the  fact 
that  subjects  probed  and  consulted  equally  in  the  equal  cost  condition,  and  it  was  speculated 
that  perhaps  this  particular  blend  of  information  sources  provided  the  best  information  supplied 
by  probing  or  consulting.  We  feel  a  similar  explanation  holds  for  workload.  Because  cost 
favors  neither  probing  or  consulting  in  the  equal  cost  condition,  participants  feel  free  to  probe 
when  they  think  a  probe  will  help  (i.e.,  provide  the  most  useful  information)  and  consult  when 
they  think  consulting  will  yield  the  best  aid.  In  contrast,  situations  where  one  source  of  infor¬ 
mation  favored  by  cost  over  the  other,  participants  feel  compelled  to  turn  to  the  cheaper  source, 
even  when  they  feel  the  more  expensive  source  might  be  more  useful.  This  cognitive  conflict 
undoubtedly  creates  some  stress  and  hence  higher  workload.  In  conditions  that  are  free  of  cost 
considerations,  subjects  feel  they  can  turn  to  whichever  source  will  be  most  useful;  this  in  turn 
creates  less  stress  and  lower  workload. 
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Figure  4-18.  Workload  as  a  Function  of  Cost  and  Stress  for  the  Civilian  Sample. 


Consultant  expertise  does  not  appear  to  affect  workload  when  stress  is  high,  as 
depicted  in  Fig.  4-19.  This  figure  also  shows  that  in  low-stress  situations  marginally  higher 
workload  rating  are  given  when  the  consultant  is  a  better  expert  (p  <  .07).  In  this  case,  higher 
workload  may  be  due  to  feeling  obligated  to  ask  the  consultant,  because  he  is  a  better  expert, 
even  when  one  really  does  not  wish  additional  information  or  feels  a  probe  would  yield  more 
useful  information. 
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Figure  4-19.  Workload  as  a  Function  of  Expertise  and  Stress. 
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As  expected,  discriminability  has  a  significant  impact  on  workload  regardless  of  stress 
level.  Under  both  high-  and  low- stress  levels,  higher  workload  ratings  are  obtained  in  the 
difficult-discrimination  condition  (ps  <  .008  and  .003,  respectively)  because  it  is  (in  fact)  more 
difficult  to  discriminate  enemy  from  friendly  submarines  when  their  respective  pump  noise 
signatures  are  only  half  a  standard  deviation  apart 

The  joint  effect  of  expertise  and  discriminability  does  impact  workload,  but  only  under 
conditions  of  low  stress  (p  <  .04).  In  Fig.  4-20  we  see  that  when  discrimination  is  easy, 
reported  workload  is  about  the  same  in  the  equal  expertise  and  consultant-a-better-expert 
conditions.  When  discrimination  is  difficult,  higher  workload  is  reported  when  the  consultant 
is  a  better  expert  than  when  the  consultant  and  decisionmaker  are  equal  in  expertise. 
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Figure  4-20.  Workload  as  a  Function  of  Expertise  and  Stress  for  Civilian  Sample. 


The  marginally  significant  result,  reported  earlier  showing  higher  workload  when  stress 
is  low  and  the  consultant  is  a  better  expert  is  now  better  defined.  It  is  when  discriminations  are 
difficult  and  subjects  feel  more  compelled  to  ask  for  additional  information,  that  subjects  report 
higher  workload.  We  now  surmise  that  two  sources  of  pressure  or  stress  may  be  aroused  and 
feed  feelings  of  workload.  One  arises  from  the  demand  to  ask  the  consultant  for  help  when 
he  is  a  better  expert  (even  in  conditions  where  one  might  not  wish  to),  and  the  other  from 
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expending  extra  effort  to  obtain  and  integrate  additional  needed  information.  It  is  not  possible 
to  discriminate  between  these  two  sources,  but  either  leads  to  higher  reported  workload. 

4.3  CIVILIAN  AND  MILITARY  DECISIONMAKERS:  COMPARISON  OF 

DECISION  STRATEGIES 

4.3.1  Premise 

As  we  gathered  our  data  about  information-seeking  and  decisionmaking  strategies,  we 
contemplated  a  question  pertinent  to  issues  of  external  validity:  would  trained  decisionmakers, 
particularly  those  with  military  experience,  act  the  same  way  as  our  samples  drawn  from 
industry  and  academia?  After  all,  it  is  our  intention  to  generalize  our  findings  to  a  military 
population.  In  an  attempt  to  address  this  issue,  a  military  sample  was  obtained  from  the  Naval 
War  College,  Newport,  RI.  In  most  regards,  we  expect  the  Navy  sample  to  act  like  our 
civilian  sample.  There  are  several  predicted  differences,  however.  Based  on  discussions  with 
Capt.  Frank  Snyder  (Ret.),  a  faculty  member  at  the  Naval  War  College,  we  hypothesize  that 
Navy  decisionmakers  will  be  willing  to  make  decisions  with  less  information  than  the  civilian 
sample.  This  is  predicted  because  military-experienced  decisionmakers  continually  must  make 
decisions  based  on  insufficient  information.  Thus,  they  are  used  to  it  and  become  conditioned 
to  work  in  this  manner.  We  also  expected  that  military  decisionmaking  experience  would  help 
decisionmakers  deal  with  stress.  Thus,  we  predicted  that  our  Navy  sample  would  be  less 
affected  by  stress  than  our  civilian  sample. 

4.3.2  Comparison  of  Information-Seeking  Strategies  Under  Stress 

The  first  analyses  explore  the  differences  in  information  seeking  for  the  military  and 
civilian  samples.  Figure  4-21  shows  that  civilians  requested  information  equally  in  high-  and 
low-stress  situations,  while  the  military  sample  showed  a  marginal  tendency  to  request  addi¬ 
tional  information  more  under  low  stress  (p  <  .09).  Perhaps,  more  germane,  it  was  hypothe¬ 
sized  that  under  stressful  conditions,  military  decisionmakers  would  seek  less  information  than 
the  civilian  decisionmakers.  This  hypothesis  is  strongly  confirmed,  as  shown  in  Fig.  4-22. 
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Figure  4-21.  Information  Request  as  a  Function  of  Stress  for  Both  Samples. 
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Figure  4-22.  Information  Seeking  Under  High  Stress  for  Military  and  Civilian  Samples. 

In  the  stress  condition,  military  personnel  requested  additional  information  only  48  percent 
of  the  time,  while  civilian  decisionmakers  asked  for  more  information  58  percent  of  the  time 
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(2  <  .001).  When  the  discrimination  condition  was  difficult  in  addition  to  stressful,  the  dif¬ 
ference  is  more  dramatic,  as  shown  in  Fig.  4-23.  Military  decisionmakers  requested  infor¬ 
mation  5 1  percent  of  the  time,  but  civilians  elicited  more  information  74  percent  of  the  time 
(£<  .001). 
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Figure  4-23.  Information  Request  Under  Stress  and  Difficult  Discriminations 
for  Military  and  Civilian  Samples. 

We  speculate  that  the  specific  training,  as  well  as  experience,  afforded  military  deci¬ 
sionmakers  has  often  forced  them  to  make  decisions  with  a  paucity  of  information.  Thus, 
they  have  developed  a  strategy  and  a  willingness  to  make  decisions  with  a  limited  amount  of 
information.  A  bromide  we  heard  quoted  about  Navy  commanders  sums  it  up,  "Maybe  incor¬ 
rect  but  never  indecisive." 

Cost  appears  to  affect  information  seeking  differently  in  the  two  samples.  Civilians 
requested  the  most  information  (63  percent)  when  the  costs  of  probing  and  consulting  were 
equal  (see  subsection  4.2.1).  Military  decisionmakers  show  an  opposite  trend  by  requesting 
additional  information  least  often  (46  percent)  in  the  equal-cost  condition  (p  <  .01 ).  Military 
personnel,  on  the  other  hand,  sought  information  the  most  (55  percent)  when  the  consultant 
cost  less  than  the  probe,  while  civilians  requested  the  least  amount  of  information  (55  percent) 
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in  that  cost  condition.  Results  show  that  high  stress  flattened  the  differences  among  the  three 
cost  conditions  for  the  civilians. 

Discriminability  appears  to  have  less  effect  on  information  seeking  for  military  deci¬ 
sionmakers  than  for  the  civilians.  Civilians  increased  their  information  seeking  by  60  percent 
in  the  difficult-discrimination  situation,  but  the  military  sample  exhibited  only  a  marginally  sig¬ 
nificant  18  percent  increase  in  the  same  condition  (p  <  .08).  The  pattern  to  seek  more  informa¬ 
tion  in  difficult-,  as  compared  to  easy-discrimination  conditions  is  essentially  unaffected  by 
stress  in  both  samples.  However,  within  the  difficult-discrimination  condition,  stress  causes  a 
5  percent  increase  in  information  seeking  for  civilians  and  a  12  percent  decrease  in  information 
seeking  for  military  decisionmakers.  We  are  not  sure  why  the  combination  of  stress  and  diffi¬ 
cult  discrimination  causes  the  military  decisionmakers  to  decrease  their  information  seeking. 
Perhaps  the  combination  of  these  two  factors  further  reinforces  their  preexisting  tendency  to 
make  decisions  with  the  information  at  hand. 

The  joint  effect  of  cost  and  expertise  significantly  impacts  information  seeking  in  the 
military  sample  (p  <  .02),  but  not  for  the  civilians.  As  shown  in  Fig.  4-24,  when  the  probe 
is  least  expensive,  military  decisionmakers  request  more  information  in  the  equal-expertise 
condition  than  the  consultant-a-better-expert  condition,  while  more  information  is  requested  in 
the  consultant-a-better-expert  condition  than  the  equal-expertise  condition  when  the  probe  is 
most  costly.  This  is  the  pattern  seen  under  low  stress,  and  although  mitigated  to  some  extent, 
the  same  pattern  is  seen  under  high  stress.  Stress,  however,  seems  to  affect  the  equal-cost 
condition  differentially.  Under  low  stress,  more  information  is  requested  when  the  consultant 
is  a  better  expert,  but  the  obverse  is  true  when  stress  is  high. 

It  was  expected  that  participants  would  request  more  information  when  the  consultant  is 
cheaper  than  the  probe  and  also  a  better  expert.  Subjects  can  get  good  information  at  low  cost. 
This  does  not  appear  true  for  the  civilians.  We  think  this  is  because  the  civilians  were  biased 
against  asking  the  consultant.  Military  subjects  show  a  propensity  to  request  more  information 
when  the  probe  is  less  expensive  than  the  consultant  and  the  consultant  is  equal  in  expertise  to 
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Figure  4-24.  Percent  Information  Request  as  a  Function  of  Cost  and  Expertise 
for  the  Military  Sample. 


the  decisionmaker,  and  this  is  because  military  decisionmaker  tends  to  probe  under  such 
conditions.  It  is  unclear  why  the  civilians  do  not  follow  suit. 

The  military  sample's  pattern  of  results,  depicted  in  Fig.  4-25,  for  the  effects  of  cost, 
discriminability,  and  stress  differs  in  several  ways  from  the  civilian’s  pattern  shown  earlier  in 
Fig.  4-10.  For  the  civilian  sample,  stress  appeared  to  increase  information  requests  in  difficult- 
versus  easy-discrimination  conditions.  The  opposite  is  true  for  the  military  sample,  where  high 
stress  seems  to  mitigate  the  differences  between  difficult-  and  easy-discrimination  conditions. 
For  military  decisionmakers,  the  largest  difference  in  information  requests  occurs  in  the  equal- 
cost  condition  (under  both  low  and  high  stress),  while  for  the  civilian,  the  difference  in  infor¬ 
mation  requests  is  about  the  same  in  probe-less-expensive  and  equal-cost  conditions.  One 
similarity  between  the  samples:  the  difference  between  difficult-  and  easy-discrimination  con¬ 
ditions,  as  to  information  requested,  is  smallest  in  the  probe-more-expensive-than-consultant 
cost  condition. 

Once  again,  we  see  that  the  military  decisionmakers'  response  to  stress  and  difficulty 
is  to  reduce  the  amount  of  information  seeking.  This  is  exactly  opposite  to  civilian  deci¬ 
sionmakers,  but  is  consonant  with  our  described  beliefs  concerning  the  training  of  military 
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Figure  4-25.  Percent  Information  Requests  as  a  Function  of  Cost,  Discriminability, 
and  Stress  for  Military  Decisionmakers. 


52 


TR-454 


ALPHATECH,  INC. _ 

commanders.  Why  military  decisionmakers  request  more  information  in  difficult  discrimi¬ 
nation  conditions,  when  the  cost  of  probing  is  equal  to  that  of  consulting,  is  more  difficult  to 
explain.  We  will  delay  the  discussion  of  this  effect  until  the  patterns  of  probing  and  consulting 
are  examined  more  closely. 

The  combined  effects  of  expertise  and  discriminability  did  not  influence  civilian  deci¬ 
sionmakers'  information  seeking  regardless  of  stress  level.  This  is  also  true  for  military  deci¬ 
sionmakers  in  low-stress  situations,  but  under  high-stress  situations  an  interesting  pattern 
develops.  As  Fig.  4-26  shows,  under  high  stress  more  information  seeking  occurs  in  the 
easy-  versus  difficult-discrimination  condition  with  expertise  equal,  while  the  opposite  is  true 
in  the  consultant-a-better-expert  situation.  Moreover,  in  this  latter  expertise  condition,  high 
stress  appears  to  increase  the  difference  between  easy-  and  difficult-  discrimination  conditions 
by  differentially  reducing  the  number  of  information  requests  in  easy  discriminations.  So 
although  high  stress  still  has  an  overall  decreasing  effect  on  information  seeking  in  the  military 
sample,  the  effect  is  considerably  less  in  the  difficult-discrimination  condition  when  the  con¬ 
sultant  is  a  better  expert.  At  this  expertise  level,  high  stress  appears  to  have  decreased  infor¬ 
mation  requests  more  in  the  easy-discrimination  situation  than  has  been  previously  observed. 
We  feel  this  is  due  to  the  willingness  of  the  military  decisionmakers  to  turn  to  the  consultant 
when  seeking  information,  particularly  when  the  consultant  is  a  better  expert  than  themselves. 
This  will  be  taken  up  again  when  the  pattern  of  probing  and  consulting  is  examined. 

4.3.3  Comparison  of  Probing  and  Consulting  Patterns 

Overall,  civilians  showed  a  preference  for  probing  over  consulting  (58  percent  versus 
42  percent,  respectively),  while  military  decisionmakers  showed  the  opposite  trend  consulting 
57  percent  of  the  time.  In  high-stress  conditions,  civilians  tended  to  consult  more  than  they 
do  in  low-stress  conditions,  but  stress  did  not  influence  information  source  for  the  military 
sample. 
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Figure  4-26.  Percent  Information  Requests  as  a  Function  of  Expertise,  Discriminability, 
and  Stress  for  Military  Sample. 

As  we  saw  before,  the  information-seeking  strategies  of  military  decisionmakers  are 
unaffected  by  stress  alone.  When  they  do  seek  information,  however,  they  favor  the  con¬ 
sultant.  This  may  be  due,  in  part,  to  their  long  experience  and  training  working  in  teams. 

The  effects  of  cost  and  expertise  on  information  source,  regardless  of  stress  level, 
appear  the  same  for  both  samples  (see  subsection  4.2.2  for  a  description).  Discriminability 
shows  some  variability  and  interacts  with  stress  differentially  for  the  two  samples.  Civ  ilians 
tended  to  probe  more  in  the  easy-  versus  difficult-discrimination  situation,  but  military  per¬ 
sonnel  showed  no  such  difference.  The  difference  in  probing  preference  evidenced  by  the 
civilians  became  stronger  under  high  stress.  Under  low  stress,  no  difference  in  information 
source  existed.  Once  again,  none  of  these  effects  observed  for  civilians  are  evident  in  the 
military  sample.  Military  decisionmakers  do  not  significantly  favor  probing  or  consulting  as 
a  function  of  stress  and  discriminability. 
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Examining  the  combined  effects  of  cost  and  expertise,  it  is  again  clear  that  military 
decisionmakers  consult  more  in  about  every  condition;  otherwise  the  pattern  of  results  for 
these  independent  variables  is  the  same  for  the  two  samples.  Essentially,  this  pattern  shows 
increased  consulting  with  increases  in  the  cost  of  probing,  and  this  is  true  for  high-  and  low- 
stress  situations.  The  other  remaining  independent  variable  combinations  did  not  produce 
significant  effects  for  either  sample. 

4.3.4  Comparison  of  Performance 

Across  all  conditions,  the  final  reward  scores  for  military  and  civilian  decisionmakers 
are  almost  identical  (about  75,  on  a  0-to-100  scale).  Civilians,  however,  show  better  perfor¬ 
mance  (as  described  previously)  under  high  versus  low  stress,  and  military  decisionmakers 
show  the  opposite  tendency,  performing  better  under  low  than  high  stress  (p  <  .04,  one  tail). 

Under  both  high-  and  low-stress  levels,  civilian  performance  peaked  in  the  equal-cost 
condition  compared  to  the  other  two  cost  conditions  (see  subsection  4.2.3).  Information  cost 
appears  to  have  no  effect  on  the  performance  of  military  decisionmakers,  and  this  is  true  under 
both  levels  of  stress. 

Expertise  impacts  performance  the  same  way  for  both  samples.  That  is,  under  high 
stress,  higher  reward  scores  are  attained  when  the  consultant  is  a  better  expert  than  when  the 
consultant  and  decisionmaker  are  equal  in  expertise,  while  under  low  stress,  higher  perfor¬ 
mance  occurred  in  the  equal-expertise  condition  than  in  the  consultant-a-better-expert  condition. 
We  explained  this  outcome  for  the  civilians  by  noting  their  probing  and  consulting  pattern  and 
how  this  pattern  interacted  with  stress  (see  subsection  4.2.2).  Essentially,  the  higher  probing 
rate  observed  in  the  equal-expertise  condition  provides  more  useful  information  under  low 
stress,  and  the  higher  consulting  rate  seen  in  the  consultant-a-better-expert  condition  supplies 
easier-to-use  information  for  the  high-stress  situations. 
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The  military  and  civilian  samples'  patterns  of  performance  for  discriminability  by 
stress  are  the  same.  Specifically,  higher  performance  occurs  in  the  easy-  versus  difficult- 
discrimination  treatment,  regardless  of  stress  level. 

The  performances  of  military  and  civilian  personnel  do  not  appear  to  be  affected  the 
same  way  by  the  joint  effects  of  cost,  expertise,  and  stress.  Figure  4-27  shows  the  perfor¬ 
mance  of  military  decisionmakers  as  a  function  of  cost  and  expertise  under  low  stress.  When 
the  probe  costs  less  than  consulting,  military  personnel  perform  better  in  the  consultant-a¬ 
better-expert  versus  equal-expertise  condition.  Civilians,  on  the  other  hand,  showed  no  differ¬ 
ence  in  performance  for  the  same  treatments  (see  Fig.  4-15,  subsection  4.2.3).  From  earlier 
discussions,  we  recall  that  civilians  probed  very  heavily  in  both  expertise  conditions,  while 
military  decisionmakers  probed  only  half  as  much  in  the  condition  where  the  consultant  is  a 
better  expert  In  fact,  military  personnel  consulted  twice  as  often  as  civilians  in  this  condition.  « 
A  probe  yields  richer  information,  but  the  decisionmaker  must  be  able  to  utilize  it  all  to  get  the 
full  benefits.  Perhaps  the  increased  amount  of  easier-to-use  processed  information  provided 
by  the  expert  consultant  enabled  the  better  performance  of  military  decisionmakers  in  the 
consul tant-a-better-expert  condition  versus  the  equal-expertise  condition. 

Figure  4-27  also  shows  that  military  decisionmakers  perform  better  in  the  equal  cost 
versus  the  probe-more-expensive  treatment  in  the  consultant-a-better-expert  condition.  Perfor¬ 
mance  increases  for  the  military  personnel  for  both  these  cost  levels  in  the  equal  expertise  con¬ 
dition.  The  increase,  however,  is  steeper  for  the  probe-more-costly  level;  thus  the  difference 
in  performance  for  the  two  cost  conditions  is  "washed  out"  in  the  equal-expertise  situation. 
Interestingly,  this  increase  in  performance  parallels  an  increase  in  consulting.  It  would  appear 
that  the  consultant's  processed  information  is  being  more  efficiently  applied  than  the  probe’s 
information.  High  stress  levels  reduce  to  nonsignificance  all  the  differences  seen  in  low  stress, 
and  this  is  true  for  both  samples. 

The  combined  effects  of  cost  and  discriminability  do  not  appear  to  impact  the  perfor¬ 
mance  of  either  sample  when  stress  is  low.  In  high-stress  situations  and  difficult-discrimination 
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Figure  4-27.  Reward  Score  and  Function  of  Cost,  Expertise,  and  Low  Stress 
for  Military  Sample. 


conditions,  civilians  performed  best  in  the  equal-cost  condition  compared  to  the  other  two 
cost  conditions.  Moreover,  the  difference  in  performance  between  the  difficult-  and  easy- 
discrimination  situations  for  equal  cost  is  half  of  what  it  is  in  the  other  two  cost  conditions. 
The  performance  of  military  decisionmakers  appeared  unaffected  by  cost  and  discriminability 
under  high  stress. 

For  the  civilian  sample  expertise,  discriminability,  and  stress  significantly  affected 
performance.  No  effects  of  these  factors  were  found  for  performance  in  the  military  sample. 
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SECTION  5 

SUMMARY,  CONCLUSIONS,  AND  RECOMMENDATIONS 
5.1  MAJOR  RESULTS 

It  was  hypothesized  that  decisionmakers  in  the  current  study  and  the  earlier  TOSCA-I 
study  would  choose  between  probing  and  consulting  according  to  a  cost-benefit  consideration. 
Cost  of  information  would  be  the  most  important  determinant  of  choice,  followed  by  consultant 
expertise.  Thus,  if  decisionmakers  and  consultants  were  equal  experts,  decisionmakers  would 
select  the  lest  costly  information  source;  if  the  cost  of  probing  and  consulting  were  equal,  then 
decisionmakers  would  seek  information  from  the  superior  consultant;  and  if  cost  and  expertise 
were  both  equal,  then  decisionmakers  would  probe,  because  a  three-digit  measurement  pro¬ 
vides  more  bits  of  information  than  a  single  opinion.  Results  of  the  current  study  match  those 
of  the  earlier  TOSCA-I  and  show  that  decisionmakers  closely  follow  these  cost-benefit  decision 
rules.  To  strive  to  obtain  the  best  information  at  the  lowest  cost  is  obviously  a  wise  course  of 
action  and  parallels  what  an  optimal  decisionmaker  would  do. 

Based  on  the  findings  of  TOSCA-I,  it  was  also  hypothesized  that  decisionmakers 
would  seek  more  information  in  difficult  versus  easy  conditions.  Strong  support  for  this 
hypothesis  is  found  in  the  data.  In  low-stress  situations  decisionmakers  significantly  sought 
more  information  in  difficult  discrimination  conditions.  Under  high  stress,  this  tendency 
became  even  stronger.  Also  as  hypothesized,  in  high-stress  and  difficult-discrimination 
situations  when  decisionmakers  requested  information,  they  turned  more  often  to  the  easier- 
to-use  processed  information  of  the  consultant  than  the  probe's  raw  measurement. 

Another  factor  that  influences  this  proclivity  of  information  seeking  is  information  cost. 
In  difficult,  high-stress,  and  cheap-probe  conditions,  decisionmakers  sought  additional  inior- 
mation  about  80  percent  of  the  time.  All  this  information  seeking,  however,  does  not  always 
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bear  fruit.  The  performance  results  show  that  heavy  probing  or  consulting  did  not  always  yield 
the  best  outcomes.  Often  the  equal-cost  condition,  where  decisionmakers  tended  to  probe  and 
consult  somewhat  equally,  yielded  the  best  performance  scores.  This  finding  seems  to  indicate 
that  decisionmakers  can  not  always  derive  or  use  all  the  information  that  is  offered  by  a  probe's 
measurement,  sometimes  they  probe  when  they  should  seek  the  easier-to-use  processed  infor¬ 
mation  of  the  consultant,  and  sometimes  they  ask  for  information  in  situations  where  additional 
information  is  not  going  to  be  very  helpful  (e.g.,  asking  the  consultant  in  difficult  discrimi¬ 
nation  conditions,  where  the  consultant's  information  is  not  better  than  the  decisionmakers'). 

In  short,  decisionmakers  tend  to  request  information  in  situations  where  it  is  not  wise  or  very 
useful  and  are  not  very  optimal  in  extracting  or  using  the  information  supplied. 

When  considering  whether  or  not  to  request  information,  decisionmakers  seemid  more 
reluctant  to  ask  for  information  when  the  consultant  was  cheapest  than  when  the  probe  was 
cheapest.  In  fact,  decisionmakers  did  not  request  more  information  in  cases  where  the  con¬ 
sultant  was  cheaper  than  the  probe  and  a  better  expert  than  the  decisionmaker.  It  seems  as  if 
decisionmakers  are  reluctant  to  give  up  control  of  the  decisionmaking  process.  That  is,  if  a 
decisionmaker  makes  a  decision  without  requesting  additional  input,  or  asks  for  another  mea¬ 
surement  that  is  then  interpreted  by  them  and  integrated  into  their  position,  decision  outcomes 
(particularly  successes)  are  personally  attributed.  If  the  consultant  is  asked,  then  the  decision¬ 
maker  must  share  the  success  with  the  consultant. 

Once,  however,  the  decision  is  made  to  request  additional  information,  much  of  this 
bias  against  asking  the  consultant  dissipates.  The  cost  by  expertise  by  level  of  stress  inter¬ 
action  clearly  shows  that  decisionmakers  turn  to  the  consultant  when  he  is  cheaper  than  the 
probe  or  a  better  expert  than  him,  or  both.  Under  high  stress,  it  is  also  clear  that  decision¬ 
makers  place  a  higher  premium  on  the  consultant's  being  expert  than  being  cheap. 

Unexpectedly,  it  was  found  that  decisionmakers  performed  better  under  high  than  low 
stress.  It  soon  became  clear,  however,  that  performance  was  only  better  in  high  stress  when 
discrimination  conditions  were  easy.  Apparently,  stress  possesses  arousal  stimuli  that  motivate 
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the  decisionmakers  to  be  more  vigilant  and  work  harder.  Under  easy  discrimination,  more 
vigilance  or  harder  work  can  lead  to  better  performance  more  easily  than  under  difficult- 
discrimination  conditions. 

It  was  hypothesized  that  military  decisionmakers  will  request  less  information  than 
civilians  when  under  stress.  Results  strongly  confirm  this  expectation.  Under  high  stress, 
civilian  decisionmakers  requested  additional  information  21  percent  more  often  than  military 
decisionmakers.  If  conditions  were  both  stressful  and  difficult  to  discriminate,  civilians  asked 
for  information  65  percent  more  often  than  military  personnel.  It  would  seem  that  the  training 
and  experience  afforded  military  commanders  has  conditioned  them  to  work  with  the  informa¬ 
tion  at  hand  and  to  request  additional  information  sparingly. 

Military  decisionmakers  did  not  favor  the  same  information  source  as  civilians. 
Civilians  showed  a  decided  preference  to  probe,  while  military  personnel  preferred  to  ask 
the  consultant.  Moreover,  in  high-stress  conditions,  civilians  tended  to  consult  more  than 
they  did  in  low-stress  conditions,  but  stress  (or  discriminability)  did  not  influence  information 
source  for  the  military  sample.  This  tendency  for  military  personnel  to  favor  the  consultant 
over  probing  may  be  due,  in  part,  to  their  long  experience  and  training  working  in  teams. 

Overall,  civilians  tended  to  perform  slightly  better  than  the  military  decisionmakers. 
This  is  probably  due  to  the  greater  amount  of  information  civilians  requested.  There  is, 
however,  some  evidence  that  military  decisionmakers  used  the  different  information  sources 
more  efficiently:  tending  to  probe  when  the  probe  would  be  more  helpful,  and  consult  when 
consulting  appeared  it  would  be  more  useful. 

5.1.1  Conclusions 

Clearly,  stressful  situations  changes  the  way  decisionmakers  strive  to  reach  decisions. 
Stress  alters  the  information-gathering  strategies  they  use  and  affects  their  decision 
performance.  Perhaps  one  of  the  clearest  findings  is  that,*under  stress  and/or  a  difficult- 
discrimination  condition,  decisionmakers  (civilians)  increase  their  requests  for  additional 
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information  to  help  reach  a  decision.  It  is  also  evident  from  the  results  that  decisionmakers 
afforded  military  training  and  experience  behave  differently  from  military-naive  decision¬ 
makers.  Military  decisionmakers  did  not  show  the  tendency  to  request  additional  information 
in  stressful  and  difficult  situations.  Instead,  they  tended  to  make  do  with  the  information  they 
had.  If  researchers  wish  to  study  the  information-seeking  and  decisionmaking  processes  of 
military  decisionmakers,  these  results  argue  that  military-experienced  personnel  should  serve 
as  subjects. 

5.2  RECOMMENDATIONS 

5.2.1  Time  Analysis 

Extensive  timing  data  were  recorded  throughout  the  experiment.  For  example,  the 
lengths  of  time  it  took  subjects  to  decide  if  they  wished  additional  information,  to  select  which 
information  source  to  choose,  and  to  integrate  all  available  information  and  come  up  with  a 
decision  regarding  submarine  hostility  were  recorded.  Unfortunately,  current  limits  in  time  and 
resources  preclude  analysis  of  these  data.  We  hope  some  time  in  the  future  to  perform  a  thor¬ 
ough  analysis  of  the  timing  data  .  We  are  encouraged  to  do  this  by  preliminary  analyses  that 
indicate  that:  1)  various  experimental  treatments  affected  the  timing  measures  differently  and 
2)  the  two  population  samples  differ  as  to  the  amount  of  time  taken  to  reach  certain  decisions. 
The  results  patterns  are  likely  to  be  quite  complex,  and  extensive  work  will  be  required  to 
extract  clear  findings  and  allow  unambiguous  interpretations.  We  feel,  however,  the  effort  will 
be  worth  it,  as  further  insights  into  decisionmaking  under  stress  may  be  gleaned. 

5.2.2  Improved  Manipulation  of  Stress- Inducing  Factors 

While  the  stress  manipulations  were  effective  in  significantly  influencing  information¬ 
seeking  behavior,  they  did  not  generate  very  high  levels  of  stress  (or  workload).  The  sub¬ 
jective  workload  ratings  provided  by  our  subjects  as  a  result  of  the  imposed  time  limit  and 
secondary  task  were  at  the  lower  end  of  the  workload  scale.  The  mean  subjective  workload 
reported  was  32  percent,  while  under  the  most  difficult  and  stressful  condition  (high  stress  and 
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difficult  discrimination),  the  mean  workload  score  was  49  percent.  It  it  evident  that  we  have 
yet  to  observe  information  seeking  and  decisionmaking  under  truly  high  levels  of  stress. 

To  address  this  situation,  we  need  to  reexamine  our  secondary  task  and  develop  a  new 
form  that  is  more  demanding  and  intrusive,  yet  realistic.  Another  avenue  to  explore  is  the 
possibility  of  operationalizing  and  manipulating  several  different  stress-inducing  variables  at 
once,  instead  of  just  two.  Two  new  candidates  worthy  of  consideration  are  the  appearance  of 
unexpected  but  highly  damaging  events,  provided  through  intel  messages  or  patterns  displayed 
on  a  map,  and  the  difficulty  or  complexity  of  the  information  to  be  interpreted.  This  would 
introduce  additional  diversity  to  stressor  types  as  well. 

It  should  also  be  noted  that  the  current  study  observed  information  seeking  and  deci¬ 
sionmaking  in  a  relatively  static  environment.  We  recommend  that  the  study  of  decisionmaking 
under  stress  be  transitioned  to  a  dynamic  environment.  A  dynamic  simulation  would  be  more 
realistic,  provide  additional  avenues  to  devise  and  operationalize  stressors,  and  provide  a 
greater  variation  of  situations  to  observe  information  seeking  and  decisionmaking. 

5.2.3  Study  of  Strategies 

The  introduction  of  a  dynamic  decisionmaking  environment  will  have  another 
advantage.  Up  to  this  point,  decisionmakers  have  been  afforded  a  very  limited  repertoire  of 
strategies  from  which  to  choose.  Our  static  simulation  has  constrained  the  number  of  options 
we  can  offer  decisionmakers.  The  dynamic  environment  will  allow  for  a  greater  number  of, 
and  hopefully  more  realistic,  strategies  at  the  decisionmakers'  disposal.  We  will,  thus,  be  able 
to  observe  how  stressed  decisionmakers  really  behave. 
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APPENDIX  A 
INSTRUCTIONS 

INSTRUCTIONS  FOR  THE  TOSCA-H  SIMULATION  GAME 
TASK  AND  GOALS 

You  are  the  commander  in  charge  of  protecting  the  naval  battle  group  from  enemy 
submarines.  Nuclear  submarines  are  usually  detected  and  identified  by  their  circulator  pump 
noise.  The  reactor  aboard  a  submarine  must  be  constantly  cooled  to  prevent  melt-down;  thus, 
the  circulator  pump  is  never  shut  down.  Different  types  of  submarines  have  different  styles  of 
pumps  and,  hence,  different  pump  noises. 

Your  principle  task  is  to  decide  if  a  detected  submarine  is  hostile  or  friendly. 
Intelligence  informs  you  that  two  submarines  are  operating  in  the  area  about  the  battle  group. 
One  submarine  is  a  friendly  attack  boat  and  has  a  pump  noise  of  370  Hz.  The  other  submarine 
is  a  hostile  Victor  boat  and  has  a  pump  noise  of  330  Hz.  On  some  occasions  the  hostile  sub¬ 
marine  will  be  a  Sierra  with  a  pump  noise  of  360  Hz.  Intelligence  will  tell  you  in  each  condi¬ 
tion  whether  the  enemy  is  a  Victor  or  Sierra  class  submarine. 

As  commander  you  know  that  the  detection  sensors  are  not  perfect,  ocean  conditions 
can  inject  error,  and  the  enemy  can  take  actions  to  increase  this  error.  On  average,  however, 
you  can  expect  an  error  distribution  around  the  true  pump  measurement  that  is  zero  mean, 
normal  in  shape,  and  has  a  standard  deviation  of  20  Hz.  Figure  1  illustrates  this  graphically. 
This  means  that  68  percent  of  the  time  the  sensors  will  report  a  pump  noise  measurement  that  is 
within  plus  or  minus  20  Hz  of  the  true  pump  noise.  On  rarer  occasions  the  sensors'  report  may 
vary  even  more  from  the  true  pump  noise. 
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Specifically,  your  task  is  one  of  discrimination  and  decision  making.  For  example, 
assume  intelligence  tells  you  that  the  hostile  submarine  in  the  area  is  a  Victor  class  (with  a 
pump  noise  of  330  Hz)  and  you  receive  a  measurement  based  on  a  single  detection  of  345  Hz. 
Given  this  information  and  the  knowledge  of  the  sensors'  accuracy  you  must  decided  if  the 
detected  pump  noise  came  from  the  enemy  or  friendly  submarine  (with  a  pump  noise  of 
370  Hz). 


IMPORTANCE  OF  CORRECT  IDENTIFICATION 

The  correct  determination  of  enemy  or  friendly  submarine  is  an  important  decision  and 
should  not  be  taken  lightly.  If  you  correctly  identify  the  submarine  as  friendly,  no  prosecution 
activities  are  initiated,  you  save  resources,  and  the  friendly  submarine  is  left  to  accomplish  its 
mission.  If  you  correctly  identify  the  submarine  as  an  enemy,  then  prosecution  activities  are 
immediately  initiated  and  there  is  a  good  chance  the  enemy  will  be  located  and  deterred  (or 
destroyed)  before  it  can  do  damage. 
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On  the  other  hand,  if  you  incorrectly  identify  the  submarine  as  friendly,  then  an  enemy 
submarine  is  left  free  to  attack  and  damage  the  battle  group.  If  you  incorrectly  identify  the 
submarine  as  hostile,  then  prosecution  activities  are  initiated  against  a  friendly  submarine. 

This  is  a  needless  waste  of  valuable  scarce  resources  and  the  possibility  of  fratricide  exists. 

ADDITIONAL  INFORMATION,  COST,  AND  CONSULTANT  CAPABILITIES 

Because  these  decisions  are  important  you  have  recourse  to  additional  information,  if 
you  deem  it  necessary,  to  help  you  make  these  decisions.  Two  resources  are  available,  but  at  a 
cost.  You  can  task  the  sensors  to  take  another  measurement  or  you  can  request  the  opinion  of  a 
"consultant"  (i.e.,  another  experienced  commander  on  a  different  platform  who  receives  infor¬ 
mation  independent  of  yours)  as  to  the  identity  of  the  detected  submarine.  Based  on  the  situa¬ 
tion,  cost  is  incurred  whenever  you  choose  to  use  either  resource.  The  cost  of  tasking  the  sen¬ 
sors  can  be  greater  than,  less  than,  or  equal  to  the  cost  of  asking  the  opinion  of  the  consultant. 

On  some  occasions  the  consultant  has  better  measurements  (i.e.,  smaller  standard  devi¬ 
ation  about  the  true  pump  noise)  than  you.  In  these  cases  it  is  expected  that,  on  the  average, 
the  consultant  would  make  better  decisions  than  you.  At  other  times,  the  measurement  qualities 
for  you  and  the  consultant  are  equal.  In  these  cases  neither  of  you  has  an  advantage  over  the 
other.  Note  that  although  the  consultant  strives  to  be  perfect,  this  is  unattainable.  The  consul¬ 
tant  does  make  mistakes.  There  are  also  times  when  the  consultant  cannot  reach  a  decision, 
and  will  return  a  message  of,  "I  do  not  know." 

The  consultant  is  aware  that  you  have  asked  for  assistance  and  takes  this  fact  into 
account  when  formulating  an  opinion.  That  is,  the  consultant  does  not  know  your  specific 
pump  noise  measurement,  but  speculates  on  what  measurement  you  must  have  received  to  lead 
you  to  ask  for  assistance.  This  speculation  (or  measurement  estimate)  along  with  the  consul¬ 
tant’s  own  independent  measurement  is  combined  to  form  the  consultant's  opinion. 
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AW  ARI>  SCORE 

After  each  trial  you  will  receive  a  score  assessing  how  well  you  used  your  resources 
and  performed  the  task.  Cost  and  the  aware  score  are  related.  Everyone  starts  each  trial  with  a 
fixed  amount  of  resources.  If  no  costs  are  incurred  and  the  submarine  is  correctly  identified, 
the  award  score  is  100  percent.  If,  however,  the  submarine  is  incorrectly  identified  then  a 
portion  of  the  initial  resources  are  used-up  or  wasted.  It  is  estimated  that  an  error  consumes 
80  percent  of  the  initial  resources.  Thus,  the  award  score  is  20  percent  in  such  cases. 

Requesting  the  opinion  of  the  consultant  or  tasking  the  sensors  for  another  measure¬ 
ment  cost  either  5  or  15  percent  of  the  initial  resources  depending  on  conditions.  In  these 
instances,  if  the  submarine  is  correctly  identified  the  award  score  is  either  95  or  85  percent, 
respectively. 

If,  however,  an  error  is  made,  that  is  the  submarine  is  incorrectly  identified,  80  percent 
of  the  remaining  initial  resources  are  wasted.  In  this  instance  the  award  score  is  15  or  5 
percent,  respectively. 

Based  on  the  new  information  (either  another  measurement  of  the  pump  noise  or  the 
consultant's  opinion)  you  must  reach  a  decision  regarding  the  detected  submarine.  After  each 
determination  you  must  rate  your  confidence,  i.e.,  the  probability  that  your  decision  is  correct 
(on  a  seven  point  scale,  where  1  equals  very  low  and  7  equals  very  high  confidence  in  the 
decision).  Feedback  as  to  the  correctness  of  each  of  your  decisions  and  the  award  score  is 
given  at  the  conclusion  of  each  trial. 

TIME  LIMITS  AND  AN  ADDITIONAL  COMMUNICATION  TASK 

On  some  occasions  you  will  be  able  to  perform  this  discrimination/decisionmaking  task 
without  any  specific  time  limit  and  with  no  other  duties  to  interfere  with  performing  the  task. 

At  other  times,  conditions  will  impose  a  specific  time  limit.  On  those  occasions  a  time  bar  will 
appear  to  help  you  budget  your  time.  Budgeting  will  be  important,  because  during  these 
instances  you  will  also  be  responsible  for  another  task  in  addition  to  the  detection  task.  This 
additional  communications  task  will  require  you  to  send  another  pump  noise  reading  from  your 
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sensors,  taken  on  another  target,  to  a  different  platform.  Then,  after  some  delay,  you  will  have 
to  verify  that  the  correct  message  was  sent. 

When  the  communication  task  is  present  it  must  be  completed  along  with  the  detection 
task  in  the  time  given.  Failure  to  complete  either  task  in  the  given  time  period  will  result  in 
failure  on  both  tasks,  and  zero  award. 

Please  consider  each  trial  and  decision  carefully.  Your  performance  will  be  compared 
to  an  optimal  algorithm,  so  try  to  do  your  best  Your  work  will  be  taken  as  the  full  measure  of 
your  ability  to  do  this  task. 
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INSTRUCTIONS  FOR  THE  TOSCA-H  SIMULATOR 


PROCEDURE 

These  instructions  are  designed  to  help  you  learn  how  to  use  the  simulator  and  perform 
the  decisionmaking  task.  Please  locate  Figure  1 .  Figure  1  presents  one  of  the  displays  you 
see  while  performing  the  task. 


T  0  S  C  A  II 


Condition  Code:  2322 


Trial  Number: 


Name:  EEE2_PIL0T1 


INTEL 

Enemy  pump  noise  360  Hz 
Friendly  pump  noise  370  Hz 
Sense  ".tandard  Deviation 
Yours  20  Hz 

Consultant's  20  Hz 
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41 5  HZ 
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En. 


Fr. 
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Consult.  Reality 

Enemy  Friendly 


Pred. 
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Fr. 


95 

15 

15 

95 

Figure  1 


TOSCA-H  Display 

The  band  at  the  top  of  the  screen  displays,  from  left  to  right,  the  condition  code,  the 
trial  number,  and  your  name.  During  training,  your  name  is  replaced  by  Training.  The 
number  following  your  name  tells  what  session  you  are  currently  performing.  There  are  four 
sessions. 
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INTEL  Window 

The  window  labeled  INTEL  (upper  left  of  screen)  provides  intelligence  information. 
Information  about  the  pump  noise  signature  of  the  friendly  submarine  and  the  pump  noise 
signature  of  the  enemy  submarine  known  to  be  operating  in  the  area  are  displayed.  The 
window  also  provides  data  on  sensor  error  for  you  and  your  consultant.  This  is  the  error 
known  to  be  inherent  in  the  sensor  hardware  employed. 

The  information  displayed  in  this  window  is  crucial  to  proper  decisionmaking.  You 
should  constantly  monitor  this  window.  Moreover,  you  want  to  be  immediately  aware  when 
the  enemy  pump  noise  signature  changes  signifying  a  different  submarine  and  when  the 
standard  deviation  of  the  consultants  error  distribution  changes  indicating  a  different  sensor 
quality.  Failure  to  monitor  and  consider  the  information  displayed  in  this  window  can  lead  to 
poor  decisionmaking. 

COMMAND  AREA  Window 

The  COMMAND  AREA  window  is  located  just  below  the  INTEL  window.  In  this 
window  information  is  requested  and  decisions  issued.  The  mouse  is  used  to  do  both. 

At  the  start.  Initial  Measurement  was  black.  When  you  clicked  on  it,  it  turned  gray  and 
the  Enemy,  Friendly,  and  ?  labels  turned  black  indicating  what  you  are  to  consider  next.  If  you 
click  on  either  Enemy  or  Friendly  (as  well  as,  ?),  this  row  of  labels  turns  gray.  If  you  click  on 
?,  the  Probe  and  Consultant  labels  turn  black.  After  you  have  decided  which  type  of  informa¬ 
tion  you  desire  and  clicked  on  it,  Enemy  and  Friendly  turn  black  again. 

Following  your  Enemy /Friendly  decision  you  enter  your  confidence  that  your  decision 
is  correct.  Once  again,  click  the  mouse  on  the  numbered  boxed  that  best  reflects  your  confi¬ 
dence  in  your  decision.  Rate  your  confidence  carefully.  This  is  how  the  commander  commu¬ 
nicates  to  others  (e.g.,  his  superiors)  how  seriously  they  should  consider  his  decision.  Do  not 
transmit  misinformation.  Once  you  have  clicked  on  a  confidence  box,  the  Feedback  label  turns 
black.  Before  you  click  on  the  feedback  button,  you  can  change  your  confidence  decision  by 
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clicking  on  another  numbered  box.  When  you  click  on  Feedback  the  trial  ends  and  you  are 
given  feedback  on  how  you  did  for  that  trial. 

REWARD  Window 

This  window  on  the  right  side  of  the  screen  describes  the  cost  of  making  an  error,  the 
current  cost  of  a  probe,  and  the  current  cost  of  asking  the  consultant.  Each  of  three  2x2  tables 
gives  the  cost  and  subsequent  reward  score  when  the  information  associated  with  that  table  is 
requested.  If  you  use  only  the  initial  measurement  and  your  decision  is  correct,  that  is,  you 
choose  enemy  and  it  is  an  enemy  submarine  or  you  choose  friendly  and  it  is  an  enemy 
submarine  -  no  addition  resources  are  used,  thus  your  reward  score  is  100%.  If  you  choose 
incorrectly,  errors  cost  80%  of  your  resources,  thus  your  reward  score  is  20%. 

The  Probe  and  Consultant  tables  are  computed  in  a  similar  fashion.  Sometimes  the  cost 
of  a  probe  or  asking  the  consultant  is  5%,  other  times  15%  of  your  resources.  Assume  that 
you  ask  for  additional  information.  In  the  cases  where  probing  or  consulting  costs  5%,  and 
you  choose  correctly,  you  earn  a  score  of  95%.  If,  however,  you  choose  incorrectly,  you  earn 
a  score  of  15%.  In  cases  where  the  cost  of  probing  or  consulting  is  15%  and  you  choose 
correctly,  your  score  is  85%.  An  incorrect  choice  would  earn  you  only  5%. 

Monitor  the  information  in  this  window  carefully.  To  make  proper  decisions  you  need 
to  consider  what  additional  information  would  cost.  You  also  need  to  consider  what  cost 
differential  exists  (if  any)  between  the  two  different  information  sources  (i.e.,  probe  and 
consultant). 

Display  for  Timed  Trials 

Locate  Figure  2.  Figure  2  shows  the  display  when  trials  are  timed  and  the  communi¬ 
cation  task  must  also  be  performed.  Before  going  on  to  the  two  new  windows,  examine  the 
windows  already  described.  Note,  some  of  the  alternative  information  and  values  that  can  be 
displayed. 
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T  0  S  C  fl  II 


Condition  Code:  1211 


Trial  Number: 
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Figure  2 


TIME  LEFT  Window 

The  time  bar  appears  when  you  must  complete  the  submarine  discrimination  task  and 
the  communication  task  in  24  seconds.  The  time  bar  displays  the  number  of  seconds  remaining 
to  complete  both  tasks.  A  beep  occurs  at  four  second  intervals. 


COMMUNICATION  AREA  Window 

The  communication  task  requires  that  you  send  a  sensor  reading  from  your  sensors, 
taken  on  another  target,  to  a  different  platform  (ship).  To  transmit  the  sensor  measure  when  it 
appears,  click  on  the  "send"  button.  The  measure  disappears  and  the  send  label  goes  gray. 

You  are  transmitting  this  data  because  the  other  platform's  sensors  are  impaired. 

To  insure  that  the  measurement  was  received  correctly,  the  other  platform  send  back  to 
you  the  value  they  received.  It  takes  about  10  seconds  for  your  transmission  to  be  received  and 
for  their  transmission  back  to  you  to  be  received.  If  you  receive  the  same  measure  you  trans- 
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mitted,  you  verify  this  by  clicking  on  the  yes,  otherwise  you  click  on  no.  Note,  when  the 
measurement  is  received  for  verification  it  is  displayed  and  a  blackened  verify  button,  followed 
by  yes  or  no,  appears. 

There  are  several  issues  to  keep  in  mind  about  the  communication  task. 

1)  After  you  request  an  initial  measurement,  there  is  a  four  or  five  second  delay  before 
your  sensors  can  take  a  measurement  on  the  new  target  and  display  it. 

2)  Be  cognizant  of  the  measurement  value  you  transmit,  so  you  can  verify  it  later. 

3)  It  takes  nine  or  ten  seconds  for  you  to  transmit  a  measurement  and  to  receive  a  value 
back  to  verify. 

4)  If  you  send  the  measures  too  late  in  the  trial,  you  may  not  have  enough  time  to 
complete  the  verification  process. 

To  complete  a  timed  trial  within  the  24  second  limit,  you  must  complete  both  the 
discrimination  task  and  the  communication  task.  The  feedback  button  will  not  become  active 
(turn  black),  until  both  tasks  are  completed.  You  must  also  click  on  the  feedback  button  within 
the  allotted  time.  During  a  timed  trial  you  receive  feedback  on  the  discrimination  task  and  your 
verification  correctness. 

Note,  that  on  any  given  trial,  there  is  a  50/50  probability  that  a  detected  submarine  is 
hostile.  Please  work  quietly  by  yourself .  At  the  end  of  each  15  trial  block,  you  will  receive  a 
message  to  fill  out  a  SWAT  recording  form;  please  do  not  forget,  also  do  not  forget  to  put  the 
condition  code  at  the  stop.  After  filling  out  the  SWAT  form  turn  it  face  down  and  do  not  refer 
back. 

Start  with  TRAINING  1.  Complete  all  the  training  trials  in  this  block. 

THANK  YOU  AND  GOOD  LUCK. 
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